This f 8000 ork aims to change some of the behaviour when writing configurations to support non-alphabetically ordered fields and other niche things for my specific use case. In this case:
- it will always follow insertion order.
- it will replace
key=valueformatting tokey = valueto look nicer. - it will NOT render keys with null values.
- it will make string values always be quoted for consistency.
- implemented in plain Java with no dependencies
- supports files in three formats: Java properties, JSON, and a human-friendly JSON superset
- merges multiple files across all formats
- can load from files, URLs, or classpath
- good support for "nesting" (treat any subtree of the config the same as the whole config)
- users can override the config with Java system properties,
java -Dmyapp.foo.bar=10 - supports configuring an app, with its framework and libraries,
all from a single file such as
application.conf - parses duration and size settings, "512k" or "10 seconds"
- converts types, so if you ask for a boolean and the value is the string "yes", or you ask for a float and the value is an int, it will figure it out.
- JSON superset features:
- comments
- includes
- substitutions (
"foo" : ${bar},"foo" : Hello ${who}) - properties-like notation (
a.b=c) - less noisy, more lenient syntax
- substitute environment variables (
logdir=${HOME}/logs)
- API based on immutable
Configinstances, for thread safety and easy reasoning about config transformations - extensive test coverage
This library limits itself to config files. If you want to load config from a database or something, you would need to write some custom code. The library has nice support for merging configurations so if you build one from a custom source it's easy to merge it in.
Table of Contents generated with DocToc
- Essential Information
- Using the Library
- Using HOCON, the JSON Superset
- Miscellaneous Notes
Typesafe Config is compatible with Java 8 and above.
You can find published releases on Maven Central.
<dependency>
<groupId>com.typesafe</groupId>
<artifactId>config</artifactId>
<version>1.4.3</version>
</dependency>
sbt dependency:
libraryDependencies += "com.typesafe" % "config" % "1.4.3"
Link for direct download if you don't use a dependency manager:
Please see NEWS.md in this directory, https://github.com/lightbend/config/blob/main/NEWS.md
- Online: https://lightbend.github.io/config/latest/api/
- also published in jar form
- consider reading this README first for an intro
- for questions about the
.conffile format, read HOCON.md in this directory
NOTE: Please read Readme #Maintained-by before spending time suggesting changes to this library.
Report bugs to the GitHub issue tracker. Send patches as pull requests on GitHub.
Before we can accept pull requests, you will need to agree to the Typesafe Contributor License Agreement online, using your GitHub account - it takes 30 seconds. You can do this at https://www.lightbend.com/contribute/cla
Please see CONTRIBUTING for more including how to make a release.
The build uses sbt and the tests are written in Scala; however, the library itself is plain Java and the published jar has no Scala dependency.
import com.typesafe.config.ConfigFactory
Config conf = ConfigFactory.load();
int bar1 = conf.getInt("foo.bar");
Config foo = conf.getConfig("foo");
int bar2 = foo.getInt("bar");
See the examples in the examples/ directory.
You can run these from the sbt console with the commands project config-simple-app-java and then run.
In brief, as shown in the examples:
- libraries should use a
Configinstance provided by the app, if any, and useConfigFactory.load()if no specialConfigis provided. Libraries should put their defaults in areference.confon the classpath. - apps can create a
Confighowever they want (ConfigFactory.load()is easiest and least-surprising), then provide it to their libraries. AConfigcan be created with the parser methods inConfigFactoryor built up from any file format or data source you like with the methods inConfigValueFactory.
Objects are immutable, so methods on Config which transform the
configuration return a new Config. Other types such as
ConfigParseOptions, ConfigResolveOptions, ConfigObject,
etc. are also immutable. See the
API docs for
details of course.
There isn't a schema language or anything like that. However, two suggested tools are:
- use the checkValid() method
- access your config through a Settings class with a field for each setting, and instantiate it on startup (immediately throwing an exception if any settings are missing)
In Scala, a Settings class might look like:
class Settings(config: Config) {
// validate vs. reference.conf
config.checkValid(ConfigFactory.defaultReference(), "simple-lib")
// non-lazy fields, we want all exceptions at construct time
val foo = config.getString("simple-lib.foo")
val bar = config.getInt("simple-lib.bar")
}
See the examples/ directory for a full compilable program using this pattern.
The convenience method ConfigFactory.load() loads the following
(first-listed are higher priority):
- system properties
application.conf(all resources on classpath with this name)application.json(all resources on classpath with this name)application.properties(all resources on classpath with this name)reference.conf(all resources on classpath with this name)
The idea is that libraries and frameworks should ship with a
reference.conf in their jar. Applications should provide an
application.conf, or if they want to create multiple
configurations in a single JVM, they could use
ConfigFactory.load("myapp") to load their own myapp.conf.
Libraries and frameworks should default to ConfigFactory.load()
if the application does not provide a custom Config object. This
way, libraries will see configuration from application.conf and
users can configure the whole app, with its libraries, in a single
application.conf file.
Libraries and frameworks should also allow the application to
provide a custom Config object to be used instead of the
default, in case the application needs multiple configurations in
one JVM or wants to load extra config files from somewhere. The
library examples in examples/ show how to accept a custom config
while defaulting to ConfigFactory.load().
For applications using application.{conf,json,properties},
system properties can be used to force a different config source
(e.g. from command line -Dconfig.file=path/to/config-file):
config.resourcespecifies a resource name - not a basename, i.e.application.confnotapplicationconfig.filespecifies a filesystem path, again it should include the extension, not be a basenameconfig.urlspecifies a URL
Note: you need to pass -Dconfig.file=path/to/config-file before the jar itself, e.g. java -Dconfig.file=path/to/config-file.conf -jar path/to/jar-file.jar. Same applies for -Dconfig.resource=config-file.conf
These system properties specify a replacement for
application.{conf,json,properties}, not an addition. They only
affect apps using the default ConfigFactory.load()
configuration. In the replacement config file, you can use
include "application" to include the original default config
file; after the include statement you could go on to override
certain settings.
If you set config.resource, config.file, or config.url
on-the-fly from inside your program (for example with
System.setProperty()), be aware that ConfigFactory has some
internal caches and may not see new values for system
properties. Use ConfigFactory.invalidateCaches() to force-reload
system properties.
The substitution syntax ${foo.bar} will be resolved
twice. First, all the reference.conf files are merged and then
the result gets resolved. Second, all the application.conf are
layered over the unresolved reference.conf and the result of that
gets resolved again.
The implication of this is that the reference.conf stack has to
be self-contained; you can't leave an undefined value ${foo.bar}
to be provided by application.conf. It is however possible to
override a variable that reference.conf refers to, as long as
reference.conf also defines that variable itself.
Any two Config objects can be merged with an associative operation
called withFallback, like merged = firstConfig.withFallback(secondConfig).
The withFallback operation is used inside the library to merge
duplicate keys in the same file and to merge multiple files.
ConfigFactory.load() uses it to stack system properties over
application.conf over reference.conf.
You can also use withFallback to merge in some hardcoded values,
or to "lift" a subtree up to the root of the configuration; say
you have something like:
foo=42
dev.foo=57
prod.foo=10
Then you could code something like:
Config devConfig = originalConfig
.getConfig("dev")
.withFallback(originalConfig)
There are lots of ways to use withFallback.
Many other configuration APIs allow you to provide a default to the getter methods, like this:
boolean getBoolean(String path, boolean fallback)
Here, if the path has no setting, the fallback would be
returned. An API could also return null for unset values, so you
would check for null:
// returns null on unset, check for null and fall back
Boolean getBoolean(String path)
The methods on the Config interface do NOT do this, for two
major reasons:
- If you use a config setting in two places, the default fallback value gets cut-and-pasted and typically out of sync. This can result in Very Evil Bugs.
- If the getter returns
null(orNone, in Scala) then every time you get a setting you have to write handling code fornull/Noneand that code will almost always just throw an exception. Perhaps more commonly, people forget to check fornullat all, so missing settings result inNullPointerException.
For most situations, failure to have a setting is simply a bug to fix
(in either code or the deployment environment). Therefore, if a
setting is unset, by default the getters on the Config interface
throw an exception.
If you want to allow a setting to be missing from
application.conf in a particular case, then here are some
options:
- Set it in a
reference.confincluded in your library or application jar, so there's a default value. - Use the
Config.hasPath()method to check in advance whether the path exists (rather than checking fornull/Noneafter as you might in other APIs). - Catch and handle
ConfigException.Missing. NOTE: using an exception for control flow like this is much slower than usingConfig.hasPath(); the JVM has to do a lot of work to throw an exception. - In your initialization code, generate a
Configwith your defaults in it (using something likeConfigFactory.parseMap()) then fold that default config into your loaded config usingwithFallback(), and use the combined config in your program. "Inlining" your reference config in the code like this is probably less convenient than using areference.conffile, but there may be reasons to do it. - Use
Config.root()to get theConfigObjectfor theConfig;ConfigObjectimplementsjava.util.Map<String,?>and theget()method onMapreturns null for missing keys. See the API docs for more detail onConfigvs.ConfigObject. - Set the setting to
nullinreference.conf, then useConfig.getIsNullandConfig.hasPathOrNullto handlenullin a special way while still throwing an exception if the setting is entirely absent.
The recommended path (for most cases, in most apps) is that you
require all settings to be present in either reference.conf or
application.conf and allow ConfigException.Missing to be
thrown if they are not. That's the design intent of the Config
API design.
Consider the "Settings class" pattern with checkValid() to
verify that you have all settings when you initialize the
app. See the Schemas and Validation
section of this README for more details on this pattern.
If you do need a setting to be optional: checking hasPath() in
advance should be the same amount of code (in Java) as checking
for null afterward, without the risk of NullPointerException
when you forget. In Scala, you could write an enrichment class
like this to use the idiomatic Option syntax:
implicit class RichConfig(val underlying: Config) extends AnyVal {
def getOptionalBoolean(path: String): Option[Boolean] = if (underlying.hasPath(path)) {
Some(underlying.getBoolean(path))
} else {
None
}
}Since this library is a Java library it doesn't come with that out of the box, of course.
It is understood that sometimes defaults in code make sense. For
example, if your configuration lets users invent new sections, you
may not have all paths up front and may be unable to set up
defaults in reference.conf for dynamic paths. The design intent
of Config isn't to prohibit inline defaults, but simply to
recognize that it seems to be the 10% case (rather than the 90%
case). Even in cases where dynamic defaults are needed, you may
find that using withFallback() to build a complete
nothing-missing Config in one central place in your code keeps
things tidy.
Whatever you do, please remember not to cut-and-paste default values into multiple places in your code. You have been warned! :-)
To read and modify configuration, you'll use the
Config
interface. A Config looks at a JSON-equivalent data structure as
a one-level map from paths to values. So if your JSON looks like
this:
"foo" : {
"bar" : 42
"baz" : 43
}
Using the Config interface, you could write
conf.getInt("foo.bar"). The foo.bar string is called a path
expression
(HOCON.md
has the syntax details for these expressions). Iterating over this
Config, you would get two entries; "foo.bar" : 42 and
"foo.baz" : 43. When iterating a Config you will not find
nested Config (because everything gets flattened into one
level).
When looking at a JSON tree as a Config, null values are
treated as if they were missing. Iterating over a Config will
skip null values.
You can also look at a Config in the way most JSON APIs would,
through the
ConfigObject
interface. This interface represents an object node in the JSON
tree. ConfigObject instances come in multi-level trees, and the
keys do not have any syntax (they are just strings, not path
expressions). Iterating over the above example as a
ConfigObject, you would get one entry "foo" : { "bar" : 42, "baz" : 43 }, where the value at "foo" is another nested
ConfigObject.
In ConfigObject, null values are visible (distinct from
missing values), just as they are in JSON.
ConfigObject is a subtype of ConfigValue, where the other
subtypes are the other JSON types (list, string, number, boolean, null).
Config and ConfigObject are two ways to look at the same
internal data structure, and you can convert between them for free
using
Config.root()
and
ConfigObject.toConfig().
As of version 1.3.0, if you have a Java object that follows
JavaBean conventions (zero-args constructor, getters and setters),
you can automatically initialize it from a Config.
Use
ConfigBeanFactory.create(config.getConfig("subtree-that-matches-bean"), MyBean.class) to do this.
Creating a bean from a Config automatically validates that the
config matches the bean's implied schema. Bean fields can be
primitive types, typed lists such as List<Integer>,
java.time.Duration, ConfigMemorySize, or even a raw Config,
ConfigObject, or ConfigValue (if you'd like to deal with a
particular value manually).
The JSON superset is called "Human-Optimized Config Object
Notation" or HOCON, and files use the suffix .conf. See
HOCON.md
in this directory for more detail.
After processing a .conf file, the result is always just a JSON
tree that you could have written (less conveniently) in JSON.
- Comments, with
#or// - Allow omitting the
{}around a root object - Allow
=as a synonym for: - Allow omitting the
=or:before a{sofoo { a : 42 } - Allow omitting commas as long as there's a newline
- Allow trailing commas after last element in objects and arrays
- Allow unquoted strings for keys and values
- Unquoted keys can use dot-notation for nested objects,
foo.bar=42meansfoo { bar : 42 } - Duplicate keys are allowed; later values override earlier, except for object-valued keys where the two objects are merged recursively
includefeature merges root object in another file into current object, sofoo { include "bar.json" }merges keys inbar.jsoninto the objectfoo- include with no file extension includes any of
.conf,.json,.properties - you can include files, URLs, or classpath resources; use
include url("http://example.com")orfile()orclasspath()syntax to force the type, or use justinclude "whatever"to have the library do what you probably mean (Note:url()/file()/classpath()syntax is not supported in Play/Akka 2.0, only in later releases.) - substitutions
foo : ${a.b}sets keyfooto the same value as thebfield in theaobject - substitutions concatenate into unquoted strings,
foo : the quick ${colors.fox} jumped - substitutions fall back to environment variables if they don't
resolve in the config itself, so
${HOME}would work as you expect. Also, most configs have system properties merged in so you could use${user.home}. - substitutions normally cause an error if unresolved, but
there is a syntax
${?a.b}to permit them to be missing. +=syntax to append elements to arrays,path += "/bin"- multi-line strings with triple quotes as in Python or Scala
All of these are valid HOCON.
Start with valid JSON:
{
"foo" : {
"bar" : 10,
"baz" : 12
}
}
Drop root braces:
"foo" : {
"bar" : 10,
"baz" : 12
}
Drop quotes:
foo : {
bar : 10,
baz : 12
}
Use = and omit it before {:
foo {
bar = 10,
baz = 12
}
Remove commas:
foo {
bar = 10
baz = 12
}
Use dotted notation for unquoted keys:
foo.bar=10
foo.baz=12
Put the dotted-notation fields on a single line:
foo.bar=10, foo.baz=12
The syntax is well-defined (including handling of whitespace and escaping). But it handles many reasonable ways you might want to format the file.
Note that while you can write HOCON that looks a lot like a Java properties file (and many properties files will parse as HOCON), the details of escaping, whitespace handling, comments, and so forth are more like JSON. The spec (see HOCON.md in this directory) has some more detailed notes on this topic.
The ${foo.bar} substitution feature lets you avoid cut-and-paste
in some nice ways.
This is the obvious use,
standard-timeout = 10ms
foo.timeout = ${standard-timeout}
bar.timeout = ${standard-timeout}
If you duplicate a field with an object value, then the objects are merged with last-one-wins. So:
foo = { a : 42, c : 5 }
foo = { b : 43, c : 6 }
means the same as:
foo = { a : 42, b : 43, c : 6 }
You can take advantage of this for "inheritance":
data-center-generic = { cluster-size = 6 }
data-center-east = ${data-center-generic}
data-center-east = { name = "east" }
data-center-west = ${data-center-generic}
data-center-west = { name = "west", cluster-size = 8 }
Using include statements you could split this across multiple
files, too.
If you put two objects next to each other (close brace of the first on the same line with open brace of the second), they are merged, so a shorter way to write the above "inheritance" example would be:
data-center-generic = { cluster-size = 6 }
data-center-east = ${data-center-generic} { name = "east" }
data-center-west = ${data-center-generic} { name = "west", cluster-size = 8 }
In default uses of the library, exact-match system properties
already override the corresponding config properties. However,
you can add your own overrides, or allow environment variables to
override, using the ${?foo} substitution syntax.
basedir = "/whatever/whatever"
basedir = ${?FORCED_BASEDIR}
Here, the override field basedir = ${?FORCED_BASEDIR} simply
vanishes if there's no value for FORCED_BASEDIR, but if you set
an environment variable FORCED_BASEDIR for example, it would be
used.
A natural extension of this idea is to support several different environment variable names or system property names, if you aren't sure which one will exist in the target environment.
Object fields and array elements with a ${?foo} substitution
value just disappear if the substitution is not found:
// this array could have one or two elements
path = [ "a", ${?OPTIONAL_A} ]
By setting the JVM property -Dconfig.override_with_env_vars=true
it is possible to override any configuration value using environment
variables even if an explicit substitution is not specified.
The environment variable value will override any pre-existing value and also any value provided as Java property.
With this option enabled only environment variables starting with
CONFIG_FORCE_ are considered, and the name is mangled as follows:
- the prefix
CONFIG_FORCE_is stripped - single underscore(
_) is converted into a dot(.) - double underscore(
__) is converted into a dash(-) - triple underscore(
___) is converted into a single underscore(_)
i.e. The environment variable CONFIG_FORCE_a_b__c___d set the
configuration key a.b-c_d
Setting the value of array items from java properties or environment variables require specifying the index in the array for the value. So, while in HOCON you can set multiple values into an array or append to an array:
## HOCON
items = ["a", "b"]
items += "c"
Using java properties you specify the exact position:
-Ditems.0="a" -Ditems.1="b"
as well as with environment variables:
export CONFIG_FORCE_items_0=a
export CONFIG_FORCE_items_1=b
Values on the same line are concatenated (for strings and arrays) or merged (for objects).
This is why unquoted strings work, here the number 42 and the
string foo are concatenated into a string 42 foo:
key : 42 foo
When concatenating values into a string, leading and trailing whitespace is stripped but whitespace between values is kept.
Quoted or unquoted strings can also concatenate with substitutions of course:
tasks-url : ${base-url}/tasks
tasks-url : ${base-url}"tasks:colon-must-be-quoted"
Note: the ${} syntax must be outside the quotes!
A concatenation can refer to earlier values of the same field:
path : "/bin"
path : ${path}":/usr/bin"
Arrays can be concatenated as well:
path : [ "/bin" ]
path : ${path} [ "/usr/bin" ]
There is a shorthand for appending to arrays:
// equivalent to: path = ${?path} [ "/usr/bin" ]
path += "/usr/bin"
To prepend or insert into an array, there is no shorthand.
When objects are "concatenated," they are merged, so object concatenation is just a shorthand for defining the same object twice. The long way (mentioned earlier) is:
data-center-generic = { cluster-size = 6 }
data-center-east = ${data-center-generic}
data-center-east = { name = "east" }
The concatenation-style shortcut is:
data-center-generic = { cluster-size = 6 }
data-center-east = ${data-center-generic} { name = "east" }
When concatenating objects and arrays, newlines are allowed inside each object or array, but not between them.
Non-newline whitespace is never a field or element separator. So
[ 1 2 3 4 ] is an array with one unquoted string element
"1 2 3 4". To get an array of four numbers you need either commas or
newlines separating the numbers.
See the spec for full details on concatenation.
Note: Play/Akka 2.0 have an earlier version that supports string
concatenation, but not object/array concatenation. += does not
work in Play/Akka 2.0 either. Post-2.0 versions support these
features.
If you have trouble with your configuration, some useful tips.
- Set the Java system property
-Dconfig.trace=loadsto get output on stderr describing each file that is loaded. Note: this feature is not included in the older version in Play/Akka 2.0. - Use
myConfig.root().render()to get aConfigas a string with comments showing where each value came from. This string can be printed out on console or logged to a file etc. - If you see errors like
com.typesafe.config.ConfigException$Missing: No configuration setting found for key foo, and you're sure that key is defined in your config file, they might appear e.g. when you're loading configuration from a thread that's not the JVM's main thread. Try passing theClassLoaderin manually - e.g. withConfigFactory.load(getClass().getClassLoader())or setting the context class loader. If you don't pass one, Lightbend Config uses the calling thread'scontextClassLoader, and in some cases, it may not have your configuration files in its classpath, so loading the config on that thread can yield unexpected, erroneous results.
Currently the library is maintained against Java 8, but version 1.2.1 and earlier will work with Java 6.
Please use 1.2.1 if you need Java 6 support, though some people have expressed interest in a branch off of 1.3.x supporting Java 7. If you want to work on that branch you might bring it up on chat. We can release a jar for Java 7 if someone(s) steps up to maintain the branch. The main branch does not use Java 8 "gratuitously" but some APIs that use Java 8 types will need to be removed.
(For the curious.)
The three file formats each have advantages.
- Java
.properties:- Java standard, built in to JVM
- Supported by many tools such as IDEs
- JSON:
- easy to generate programmatically
- well-defined and standard
- bad for human maintenance, with no way to write comments, and no mechanisms to avoid duplication of similar config sections
- HOCON/
.conf:- nice for humans to read, type, and maintain, with more lenient syntax
- built-in tools to avoid cut-and-paste
- ways to refer to the system environment, such as system properties and environment variables
The idea would be to use JSON if you're writing a script to spit out config, and use HOCON if you're maintaining config by hand. If you're doing both, then mix the two.
Two alternatives to HOCON syntax could be:
- YAML is also a JSON superset and has a mechanism for adding
custom types, so the include statements in HOCON could become
a custom type tag like
!include, and substitutions in HOCON could become a custom tag such as!subst, for example. The result is somewhat clunky to write, but would have the same in-memory representation as the HOCON approach. - Put a syntax inside JSON strings, so you might write something
like
"$include" : "filename"or allow"foo" : "${bar}". This is a way to tunnel new syntax through a JSON parser, but other than the implementation benefit (using a standard JSON parser), it doesn't really work. It's a bad syntax for human maintenance, and it's not valid JSON anymore because properly interpreting it requires treating some valid JSON strings as something other than plain strings. A better approach is to allow mixing true JSON files into the config but also support a nicer format.
This may not be comprehensive - if you'd like to add mention of your wrapper, just send a pull request for this README. We would love to know what you're doing with this library or with the HOCON format.
- Typesafe Config Guice https://github.com/racc/typesafeconfig-guice
- Ficus https://github.com/iheartradio/ficus
- configz https://github.com/arosien/configz
- configs https://github.com/kxbmap/configs
- config-annotation https://github.com/zhongl/config-annotation
- PureConfig https://github.com/pureconfig/pureconfig
- Simple Scala Config https://github.com/ElderResearch/ssc
- konfig https://github.com/vpon/konfig
- ScalaConfig https://github.com/andr83/scalaconfig
- static-config https://github.com/Krever/static-config
- validated-config https://github.com/carlpulley/validated-config
- Cedi Config https://github.com/ccadllc/cedi-config
- Cfg https://github.com/carueda/cfg
- circe-config https://github.com/circe/circe-config
- args4c https://github.com/aaronp/args4c
- beamly-core.config https://github.com/beamly/beamly-core.config
- config4k https://github.com/config4k/config4k
- hoplite https://github.com/sksamuel/hoplite
- SHocon https://github.com/akka-js/shocon (Supports Scala.js and Scala Native)
- sconfig https://github.com/ekrich/sconfig (Supports JVM, Scala Native, and Scala.js)
- Manage your HOCON configuration files with Puppet!: https://forge.puppetlabs.com/puppetlabs/hocon
- https://github.com/yellowblood/hocon-js (missing features, under development)
- A web based linting tool http://www.hoconlint.com/
The license is Apache 2.0, see LICENSE-2.0.txt.
The "Typesafe Config" library is an important foundation to how Akka and other JVM libraries manage configuration. We at Lightbend consider the functionality of this library as feature complete. We will make sure "Typesafe Config" keeps up with future JVM versions, but will rarely make any other changes.
We are thankful for all the work @havocp has put into creating the library initially and supporting its users over many more years, even after leaving Lightbend.