DEPOT is a file storage and retrieval framework we created to solve the need of switching different storage systems when deploying in different environments. We wanted a unique and cohesive API that made possible to keep storing files the same way independently from where they were actually stored.
As systems evolve and change during time we also wanted the ability to switch those storages whenever required without breaking past files or changing any code. That lead to various features of DEPOT that specifically pointed this problem, the last of this list is the new Storage Aliases support.
If you used a single storage, myfiles, registered as the default one, when you wanted to switch from Local Storage to S3 that was easy, you could register a new mys3files storage on S3 and switch the default one to be that one. Your old files would continue to be served from the old storage and your new files would be uploaded and served from the new storage, because the system knew that some files were saved in myfiles and some were on mys3files.
Now, suppose you want to keep your user avatars on a separated storage from their uploaded content. You could already declare an avatars storage and just force upload of all your files on that specific storage
from depot.fields.sqlalchemy import UploadedFileField
class User(Base):
__tablename__ = 'users'
uid = Column(Integer, autoincrement=True, primary_key=True)
name = Column(Unicode(16), unique=True)
avatar = Column(UploadedFileField(upload_storage='avatars'))
This would correctly store all your user avatars on the avatars storage. But what happened when you wanted to switch your avatars storage from saving files on disk to saving them on S3?
Because in this case DEPOT knew that all your avatars were on the avatars storage, you had to put your system on maintenance, manually move all the files on S3, switch the avatars storage configuration and then restart your application. This lead to downtime and wasn’t very convenient.
Introducing the new Aliases feature you can now declare two different storages
DepotManager.configure('local_avatars', {'depot.storage_path': '/var/www/lfs'})
DepotManager.configure('s3_avatars', {'depot.backend': 'depot.io.awss3.S3Storage', 'depot.access_key_id': ...})
tell DEPOT that avatars is just an alias for local_avatars
DepotManager.alias('avatars', 'local_avatars')
and whenever you stored a file on avatars it would actually be stored on local_avatars.
Want now to switch storing your files on S3? Just switch the alias configuration
DepotManager.alias('avatars', 's3_avatars')
And all your avatars will now be stored on S3 while the old one continue to be served from the disk as DEPOT knows they are actually on local_avatars.
This is a pretty simple and convenient solution that perfectly solved our need introducing the ability to evolve your storages forever as far as you only stored files on aliases and never directly on the storages themselves.