Merging yaml config files recursively with bash
No.
Bash has no support for nested data structures (its maps are integer->string or string->string only), and thus cannot represent arbitrary YAML documents in-memory.
Use a more powerful language for this task.
Late to the party, but I also wrote a tool for this:
https://github.com/benprofessionaledition/yamlmerge
It's almost identical to Ondra's JVM tool (they're even both called "yaml merge"), the key difference being that it's written in Go so it compiles to a ~3MB binary with no external dependencies. We use it in Gitlab-CI containers.
I recommend yq -m
. yq is a swiss army knife for yaml, very similar to jq (for JSON).
You can use for example perl. The next oneliner:
perl -MYAML::Merge::Simple=merge_files -MYAML -E 'say Dump merge_files(@ARGV)' f1.yaml f2.yaml
for the next input files: f1.yaml
date:
epoch: 2342342343
format:
date_regular: "%d-%m-%Y"
f2.yaml
date:
format:
date_regular: regular dates are boring
date_special: "!!!%d-%m-%Y!!!"
prints the merged result...
---
date:
epoch: 2342342343
format:
date_regular: regular dates are boring
date_special: '!!!%d-%m-%Y!!!'
Because @Caleb pointed out that the module now is develeloper only, here is an replacement. It is a bit longer and uses two (but commonly available) modules:
- the YAML
- and the Hash::Merge::Simple
perl -MYAML=LoadFile,Dump -MHash::Merge::Simple=merge -E 'say Dump(merge(map{LoadFile($_)}@ARGV))' f1.yaml f2.yaml
produces the same as above.