By doing this, we can take advantage of py.test's ability to create
temporary directories that are then cleaned up later during testing.
This helps for sandboxing things.
This also involves a ton of changes:
- Changing the get_app stuff appropriately, getting rid of the
setup_fresh_app decorator
- Making said fixture
- Switching over a billion tests to use it