Dynamic Variables
Template substitutions¶
Dynaconf has 2 tokens to enable string substitutions, the interpolation works with @format and @jinja.
@format token¶
Dynaconf allows template substitutions for strings values, by using the @format token prefix and including placeholders accepted by Python's str.format method Dynaconf will call
it lazily upon access time.
The call will be like:
"<YOURVALUE>".format(env=os.environ, this=dynaconf.settings)
So in your string you can refer to environment variables via env object, and also to variables defined int the settings object itself via this reference. It is lazily evaluated on access it will use the final value for a settings regardless the order of load.
Example:
export PROGRAM_NAME=calculator
settings.toml
[default]
DB_NAME = "mydb.db"
[development]
DB_PATH = "@format {env[HOME]}/{this.current_env}/{env[PROGRAM_NAME]}/{this.DB_NAME}"
{env[HOME]}is the same asos.environ["HOME"]or$HOMEin the shell.{this.current_env}is the same assettings.current_env{env[PROGRAM_NAME]}is the same asos.environ["PROGRAM_NAME"]or$PROGRAM_NAMEin the shell.{this.DB_NAME}is the same assettings.DB_NAMEorsettings["DB_NAME"]
so in your program
from dynaconf import settings
settings.DB_PATH == '~/DEVELOPMENT/calculator/mydb.db'
@format token can be used together with tokens like @str, @int, @float, @bool, and @json to cast the results parsed by @format.
Example:
Casting to integer from @format templated values
NUM_GPUS: 4
# This returns the integer after parsing
FOO_INT: "@int @format {this.NUM_GPUS}"
# This returns the string after parsing
FOO_STR: "@format {this.NUM_GPUS}"
@jinja token¶
If jinja2 package is installed then dynaconf will also allow the use jinja to render string values.
Example:
export PROGRAM_NAME=calculator
settings.toml
[default]
DB_NAME = "mydb.db"
[development]
DB_PATH = "@jinja {{env.HOME}}/{{this.current_env | lower}}/{{env['PROGRAM_NAME']}}/{{this.DB_NAME}}"
so in your program
from dynaconf import settings
settings.DB_PATH == '~/development/calculator/mydb.db'
The main difference is that Jinja allows some Python expressions to be evaluated such as {% for, if, while %} and also supports calling methods and has lots of filters like | lower.
Jinja supports its built-in filters listed in Builtin Filters Page and Dynaconf includes additional filters for os.path module: abspath. realpath, relpath, basename and dirname and usage is like: VALUE = "@jinja {{this.FOO | abspath}}"
@jinja token can be used together with tokens like @str, @int, @float, @bool, and @json to cast the results parsed by @jinja.
Example:
Casting to integer from @jinja templated values
NUM_GPUS: 4
# This returns the integer after parsing
FOO_INT: "@int @jinja {{ this.NUM_GPUS }}"
# This returns the string after parsing
FOO_STR: "@jinja {{ this.NUM_GPUS }}"
Example:
Casting to a json dict from @jinja templated values
MODEL_1:
MODEL_PATH: "src.main.models.CNNModel"
GPU_COUNT: 4
OPTIMIZER_KWARGS:
learning_rate: 0.002
MODEL_2:
MODEL_PATH: "src.main.models.VGGModel"
GPU_COUNT: 2
OPTIMIZER_KWARGS:
learning_rate: 0.001
# Flag to choose which model to use
MODEL_TYPE: "MODEL_2"
# This returns the dict loaded from the resulting json
MODEL_SETTINGS_DICT: "@json @jinja {{ this|attr(this.MODEL_TYPE) }}"
# This returns the raw json string
MODEL_SETTINGS_STR: "@jinja {{ this|attr(this.MODEL_TYPE) }}"
Name aliasing¶
In certain cases you need to reuse some variable value respecting the exact data type it is previously defined in another step of the settings load.
Example:
from dynaconf import Dynaconf
settings = Dynaconf(settings_files=["settings.toml", "other.toml"])
thing = "value"
number = 42
other_thing = "@get thing"
# providing default if thing is undefined
other_thing = "@get thing 'default value'"
## type casting
# Ensure the aliased value is of type int
other_number = "@get number @int"
# Ensure type and provide default if number is undefined
other_number = "@get number @int 12"
The @get is lazily evaluated and subject to DynaconfFormatError in case of
malformed expression, it is recommended to add validators.
Reading from files¶
You can read from text files using the @read_file token. This token will read the file
and return its content as a string. The file path can be absolute or relative to the current working directory.
Example:
SECRET_KEY = "@read_file /path/to/secret_key"
echo "my_secret_key" > /path/to/secret_key
from dynaconf import Dynaconf
settings = Dynaconf(settings_files=["settings.toml"])
assert settings.SECRET_KEY == "my_secret_key"
You can also specify a default value in case the file does not exist or isn't readable.
Example:
SECRET_KEY = "@read_file /path/to/secret_key default_value"
If the file does not exist and there is no default value, a FileNotFoundError will be raised upon access.
Combining with other tokens¶
You can combine the @read_file with other interpolation tokens like @format or @jinja and @get to read files with dynamic paths or content.
Example:
FILENAME = "secret_key"
DIRECTORY = "/path/to"
SECRET_KEY = "@read_file @format {this.DIRECTORY}/{this.FILENAME}"
SECRET_KEY = "@read_file @jinja {{ this.DIRECTORY }}/{{ this.FILENAME }}"
COMPLETE_PATH = "@format {this.DIRECTORY}/{this.FILENAME}"
SECRET_KEY = "@read_file @get COMPLETE_PATH"
Combining with string utils¶
If the file needs stripping or other string manipulations, you can combine the @read_file with the string utils tokens
SECRET_KEY = "@strip @read_file /path/to/secret_key"
String utils¶
Dynaconf provides some string utils to manipulate strings. The following utils are available:
"@upper foo" == "FOO"
"@lower FOO" == "foo"
"@title foo bar" == "Foo Bar"
"@capitalize foo bar" == "Foo bar"
"@strip foo bar \n" == "foo bar"
"@lstrip foo bar \n" == "foo bar \n"
"@rstrip foo bar \n" == " foo bar"
"@split foo bar" == ["foo", "bar"]
"@casefold Foo BAR" == "foo bar"
"@swapcase Foo BAR" == "fOO bar"
@upper: Converts the string to uppercase.@lower: Converts the string to lowercase.@title: Converts the string to title case.@capitalize: Capitalizes the first character of the string.@strip: Removes leading and trailing whitespace from the string.@lstrip: Removes leading whitespace from the string.@rstrip: Removes trailing whitespace from the string.@split: Splits the string into a list of words.@casefold: Converts the string to casefolded form (for case-insensitive comparisons).@swapcase: Swaps the case of all characters in the string.
Access Hooks¶
Dynaconf supports access hooks, which allow the user to register functions that
will be called each time a variable is accessed on a Dynaconf instance.
This feature is useful to load settings from external sources, such as from cloud provider's secrets managers, or when you need special processing on top of raw values (such as decrypting an encrypted value).
NOTES:
- You must define the logic of each hook, dynaconf just calls it
- You must take care of caching and caching invalidation
- This may introduce overhead dependending on how the hook function is implemented
To register an access hook, you need to wrap the Dynaconf instance with
HookableSettings and pass a Action to Hook mapping in to the constructor
in the _registered_hooks kwarg. For example,
# So you have a function that retrieves data from external source
CACHE = {} # could be Redis
def get_data_from_somewhere(settings, result, key, *args, **kwargs):
"""Your function that goes to external service like gcp, aws, vault, etc
Parameters:
settings: A copy of settings, changes made are valid only during the
scope of this function.
result: The object that holds the setting known to Dynaconf at the
moment of access in the .value attribute.
key: The accessed key just as passed to Dynaconf.get, usually a
single str, but if the requested key was dotted then the whole
path is passed to this function.
*args and **kwargs: extra positional or named arguments passed to .get
such as default,cast,fresh,parent etc.
"""
if key in CACHE:
return CACHE[key]
# Assuming result.value is '@special:/foo/bar:token'
if isinstance(result.value, str) and result.value.startswith("@special:"):
_, path, identifier = result.value.split(":")
# value = get_value_from_special_place(path, identifier)
value = CACHE[key] = "lets believe it was retrieved from special place"
return value
return result
# ---
from dynaconf import Dynaconf
from dynaconf.hooking import HookableSettings, Hook, Action
# Create a Dynaconf instance
settings = Dynaconf(
# set the Hookable wrapper class
_wrapper_class=HookableSettings,
# Register your hooks, see the Action enum for available values
# you can have multiple hooks, value passes from one to another
_registered_hooks={
Action.AFTER_GET: [Hook(get_data_from_somewhere)]
},
)
# this value can actually come from settings files or envvars
settings.set("token", "@special:/vault/path:token")
assert settings.token == "lets believe it was retrieved from special place"
# from cache this time
assert settings.token == "lets believe it was retrieved from special place"
Merging with access hooks¶
When the result.value is a mergeable data structure like dict or list
and you want the merging to be applied, then the hook must delegate the merging
to the given settings by seting the result (which will perform the merging)
and returning the result of geting the setting from the given settings.
def get_data_from_somewhere(settings, result, key, *args, **kwargs):
"""Your function that goes to external service like gcp, aws, vault, etc
NOTE: settings in this context is copy of settings, changes made are
valid only during the scope of this function.
"""
if key in CACHE:
return CACHE[key]
...
interesting_keys = ["foo", "bar", "zaz"] # known to be dicts or lists
if key in interesting_keys:
value = CACHE[key] = get_value_from_special_place(key)
settings.set(key, value) # process merging and parsing
return settings.get(key, result.value)
return result
Registering Hooks After Dynaconf is instantiated¶
It is ok to register hooks later, which might be useful to do it conditionally.
if something:
settings.set("_registered_hooks", {Action.AFTER_GET: [Hook(....)]})
It also works if you use dynaconf_hooks.py to register it conditionally.
# dynaconf_hooks.py
def post(settings):
data = {}
if settings.something:
data["_registered_hooks"] = {Action.AFTER_GET: [Hook(....)]}
return data