Ability to use object store for images

[jpds]

Issue type

[x] Feature request
[ ] Bug report
[ ] Documentation

Environment

• NetBox version: v2.2.9

Description

I'm running Netbox in a Kubernetes cluster. It currently runs as a single container as I have a block device mounted into the container so that there's persistent storage for the image attachments which were introduced in #152.

This works OK, but I cannot run multiple instances of netbox in a clustered fashion as it's tied to the block device.

Could there be some integration with https://github.com/jschneier/django-storages so that S3 can be used as a backend for the image attachments?

[jeremystretch]

Per the discussion in PR #2466, the configuration scheme for this needs to be fleshed out. While I don't have a strong preference, whatever solution we decide on must be platform- and vendor-agnostic, and easily extendable to accommodate similar services.

[nward]

Hi, I've been directed here from #3095 - thanks for that @jeremystretch.

I have implemented something as a PoC for this.
Additionally, I implemented a way to optionally load another application for display. "django-storages" doesn't require this, as it directs the user for the application to a direct download URL. This has the effect that images are not behind the NetBox access control anymore. Writing an application for this would allow an administrator to use S3/etc. for only storage, not for serving the files.

For the configuration, I toyed with several approaches:

1. Have configuration for all the various options (S3_* etc.). This doesn't work, as in NetBox we have only specific settings passed through from configuration.py to settings.py, and we would need settings.py to support every possible option - doesn't really fly.
2. Have a dictionary defined in configuration.py which the "external" applications, storage classes etc. can use for settings. This requires that these applications, storage classes etc. are written custom for NetBox (or at the very least, with NetBox support). This is not untenable as potentially some sort of wrapper classes could be written but it's not ideal.
3. Have a dictionary defined in configuration.py which NetBox's settings.py takes and translates in to settings.py parameters.
4. Have settings in configuration.py which are found by settings.py with a prefix, and then the prefix stripped off and defined as settings.py parameters. This is OK, but a bit clunky and hard to read.

In my implementation, I opted for (3) which I found this to be a good balance between (1) and (2). It has some things I'd like to improve. Primarily reduce or eliminate the ability for a configuration parameter to be named the same as an existing NetBox parameter and override it - perhaps it needs to scan a list of known settings used by NetBox and error if the administrator attempts to pass them in this dictionary. That seems fairly easy to implement.

Perhaps (3) and (2) at the same time would be a good option - "NetBox Native" storage systems and applications could use (2) while existing systems (django-storages, etc.) could use (3).

[chaomodus]

Note I have very simply implemented the above option 3 for our use since we need to use a Swift back-end to save images (and there are numerous configuration settings that have to be imported) in case anyone is interested.

https://gerrit.wikimedia.org/r/c/operations/software/netbox/+/518785

[steffann]

I'd be happy to take this issue.

Because of the separation between settings.py and configuration.py I suggest to abstract the implementation from the configuration and start with the most common one: AWS S3 and compatible.

The configuration could look like this:

MEDIA_STORAGE = {
  'BACKEND': 'S3',
  'ACCESS_KEY_ID': 'Secret',
  'SECRET_ACCESS_KEY': 'Secret',
  'REGION_NAME': 'eu-west-1',
  'ENDPOINT_URL': 'https://example/',
  'BUCKET_NAME': 'netbox',
  'DEFAULT_ACL': 'public-read',
}

For the implementation I suggest adding django-storages as a dependency when the S3 back-end is used.

Any objections to this approach?