- Changed config files of test configs to use SQL
- Updated celery initialization tools, factored them to be able to
use the "big instance" application stuff
1. No need to drop media_data['exif'], we only have and
want media_data['exif_all'].
2. Use media['_id'] instead of media._id (better not use
dot-notation on mongo objects in such a low level tool).
As the newly created entries have media_data.exif_all use
the exif_display_iter method to calculate the exif entries
for displaying on the fly.
After the mongo migration all the old entries will also
have exif_all and will work again.
In addition to filling the normal slot media_data['exif'],
now also fill media_data.exif_all. This is the new slot
used by SQL.
For a few moments this will create duplicated entries in
the mongo db. But this shouldn't hurt.
MediaEntry.media_data.exif_all will contain all the
"clean" EXIF data.
MediaEntry.exif_display_iter() is an iterator that fetches
the most interesting entries for display from that data.
When creating a new media_data row, the new row needs to
know the MediaEntry it is associated with. I have no idea,
why this worked before at all. Maybe some implicit tricks
by sqlalchemy?
processing.py -> processing/__init__.py
This is in preparation for splitting processing a bit.
The main reason for the split is celery setup: celery needs
to be setup before even importing and importing and
subclassing some of its parts. So it's better to move the
critical parts into their own submodule and import it as
late as needed.
So all models are ready when connecting to the db and so
our "db" object has all models listed on it, create a
function to load all models from the media_types, etc. Call
it in setup_database()
Problem: This gives celery warnings, because celery is
imported before being setup properly. No idea how to fix
this now. So media-type loading is excluded from
load_models for now.
As the queries are quite verbose, disable them for now.
Reenabling them should be done in the central logging
config, which is another story for celery and bin/gmg.
Kind of useful to see but... I don't think they're needed, and I'm not
super comfortable with print statements being in migrations. Seems
semi bloated!
Bug #270 asks for a lazycelery.sh script much like lazyserver.sh. Rather
than duplicate the code, I consolidated them into a single script,
lazystarter.sh. The script reconfigures itself a bit, and runs a
particular server, based on the name that's used to call it, but no matter
what it uses the same code to offer help and find configuration files and
server launchers. Hopefully this will make it easy to add other
features/fix bugs as needed in the future, and have them stay in sync.
The mongosql tool is really dumping directly into the sql
database and is trying not to use too much logic that might
change later.
So this means, it needs to create the migration records on
its own!
So add a bunch of records with version=0.
The actual code is just a simple for loop; there might be a better
implementation but this is a fine start. I also extended test_delete to
check this too.
