26

what I want to acomplish is:

1.) Having a config file as template, with variables like $version $path (for example apache config)

2.) Having a shell script that "fills in" the variables of the template and writes the generated file to disk.

Is this possible with a shell script. I would be very thankfull if you can name some commands/tools I can accomplish this or some good links.

10 Answers 10

23

This is very possible. A very simple way to implement this would be for the template file to actually be the script and use shell variables such as

#! /bin/bash
version="1.2.3"
path="/foo/bar/baz"
cat > /tmp/destfile <<-EOF
here is some config for version $version which should
also reference this path $path
EOF

You could even make this configurable on the command line by specifying version=$1 and path=$2, so you can run it like bash script /foo/bar/baz 1.2.3. The - before EOF causes whitespace before the lines be ignored, use plain <<EOF if you do not want that behavior.

Another way to do this would be to use the search and replace functionality of sed

#! /bin/bash
version="1.2.3"
path="/foo/bar/baz"
sed -e "s/VERSION/$version/g" -e "s/PATH/$path/" /path/to/templatefile > /tmp/destfile

which would replace each instance of the strings VERSION and PATH. If there are other reasons those strings would be in the template file you might make your search and replace be VERSION or %VERSION% or something less likely to be triggered accidentally.

18

No tools necessary other than /bin/sh. Given a template file of the form

Version: ${version}
Path: ${path}

or even with mixed shell code included

Version: ${version}
Path: ${path}
Cost: ${cost}\$
$(i=1; for w in one two three four; do echo Param${i}: ${w}; i=$(expr $i + 1); done)

and a shell parsable configuration file like

version="1.2.3-r42"
path="/some/place/under/the/rainbow/where/files/dance/in/happiness"
cost="42"

it is a simple matter to expand this to

Version: 1.2.3-r42
Path: /some/place/under/the/rainbow/where/files/dance/in/happiness
Cost: 42$
Param1: one
Param2: two
Param3: three
Param4: four

Indeed, given the path to the configuration file in shell variable config_file and the path to the template file in template_file, all you need to do is:

. ${config_file}
template="$(cat ${template_file})"
eval "echo \"${template}\""

This is perhaps prettier than having complete shell script as the template file (@mtinberg's solution).

The complete naive template expander program:

#!/bin/sh

PROG=$(basename $0)

usage()
{
    echo "${PROG} <template-file> [ <config-file> ]"
}

expand()
{
    local template="$(cat $1)"
    eval "echo \"${template}\""
}

case $# in
    1) expand "$1";;
    2) . "$2"; expand "$1";;
    *) usage; exit 0;;
esac

This will output the expansion to standard output; just redirect standard output to a file or modify the above in obvious fashion to produce the desired output file.

Caveats: Template file expansion would not work if the file contained unescaped double quotes ("). For security reasons, we should probably include some obvious sanity checks or, even better, perform shell escaping transformation if the template file is generated by external entity.

9

The easiest way to do this simply in Linux CLI is to use envsubst and Environment Variables.

Example template file apache.tmpl:

<VirtualHost *:${PORT}>
    ServerName ${SERVER_NAME}
    ServerAlias ${SERVER_ALIAS}
    DocumentRoot "${DOCUMENT_ROOT}"
</VirtualHost>

Run envsubst and output result to new file my_apache_site.conf:

export PORT="443"
export SERVER_NAME="example.com"
export SERVER_ALIAS="www.example.com"
export DOCUMENT_ROOT="/var/www/html/"
envsubst < apache.tmpl > my_apache_site.conf

Output:

<VirtualHost *:443>
    ServerName example.com
    ServerAlias www.example.com
    DocumentRoot "/var/www/html/"
</VirtualHost>
  • 2
    It's part of GNU gettext, and therefore available on all major Unix platforms (including macOS X), as well as Microsoft Windows. – tricasse Mar 30 at 23:28
4

You probably ought to look into a configuration management system like Puppet or Chef. These can easily do what you describe above and much more.

  • 1
    thanks. Absolutely, I have Chef installed and running. But it adds a lot of overhead, when you have to write your own cookbooks. I do not know the ruby programming language and my conclusion was. its easier to do this with a shell script for the easier cases (if possible). – Markus Jul 6 '11 at 21:42
  • Looks like Puppet and Chef both use ERB for templates, and that's ridiculously easy to get started with. Given a variable name, the string <%= name %> in a template will get replaced with name's value. How you define name outside the template differs between the two systems, obviously. – Mike Renfro Jul 6 '11 at 23:42
  • 1
    Yes templating (With Chef) itself is absolutely easy. But using chef as Framework (and writing the cookbooks) requires a lot of time. To get the data into the templates you need to understand where and how Chef manages the "merging" of datasources and a lot of other stuff. I have started writing my own cookbooks, but a shell script would in my special case be 100 times faster... – Markus Jul 15 '11 at 18:49
  • Getting the infrastructure going for Chef or Puppet can be a pain or you can try to figure out how to run them headless which is a fun adventure. Ansible works fine in pull or push mode out of the box so it might strike a better balance of figuring it out and scripting it yourself. docs.ansible.com/template_module.html – chicks Jun 16 '15 at 13:27
4

If you want lightweight and real templates rather than shell code that generates new files, the usual choices are sed& awk or perl. Here is one link: http://savvyadmin.com/generate-text-from-templates-scripts-and-csv-data/

Me, I'd use a real language like perl, tcl, python, ruby or something else in that class. Something built for scripting. They all have good, simple templating tools and tons of examples in google.

4

I use shtpl for that. (private project of mine, which means, it is not widely in use. But maybe you want to test it anyway)

For example you want to generate an /etc/network/interfaces out of a csv-file, you can do it like that:

CSV-file content (here test.csv):

eth0;10.1.0.10;255.255.0.0;10.1.0.1
eth1;192.168.0.10; 255.255.255.0;192.168.0.1

Template (here interfaces.tpl):

#% IFS=';'
#% while read "Val1" "Val2" "Val3" "Val4"; do
auto $Val1 
iface $Val1 inet static
  address $Val2 
  netmask $Val3 
  gateway $Val4 

#% done < "$CSVFILE"

Command:

$ CSVFILE=test.csv sh -c "$( shtpl interfaces.tpl )"

Result:

auto eth0 
iface eth0 inet static
  address 10.1.0.10 
  netmask 255.255.0.0 
  gateway 10.1.0.1 

auto eth1 
iface eth1 inet static
  address 192.168.0.10 
  netmask  255.255.255.0 
  gateway 192.168.0.1

Enjoy!

1

I improved FooF's answer so that the user doesn't need to unescape double quotes manually:

#!/bin/bash
template="$(cat $1)"
template=$(sed 's/\([^\\]\)"/\1\\"/g; s/^"/\\"/g' <<< "$template")
eval "echo \"${template}\""
1

I recently published a bash script that accomplishs just that using a jinja-like template syntax. It's called cookie. Here's a demo:

cookie demo

  • How does this answer the question? – RalfFriedl Nov 14 '18 at 6:45
  • 1
    @RalfFriedl I'm not sure what you mean. I just reread the question thinking I must have missed something, but I'm not seeing it. In fact, how does this not answer any part of the question that was asked? It's almost like I built cookie with the sole purpose of answering this question... which I kindof did, though I may have worded it a bit differently in my head at the time. :) – Bryan Bugyi Nov 14 '18 at 10:54
  • I'm a little late to play mediator, but perhaps Ralf was just charitably prompting you to explain how cookie answers the question, rather than trying to uncharitably imply you don't know your own project well enough to judge whether it might solve someone's problem. – abathur Apr 25 at 0:28
0

I am probably late to this party. However, I stumbled about the very same problem and I opted for creating my own BASH template engine in a few lines of code:

Lets say that you have this file.template:

# My template
## Author
 - @NAME@ <@EMAIL@>

And this rules file:

NAME=LEOPOLDO WINSTON
EMAIL=leothewinston\@leoserver.com

You execute this command:

templater rules < file.template

You get this:

# My template
## Author
 - LEOPOLDO WINSTON <leothewinston@leoserver.com>

You can install it by:

 bpkg install vicentebolea/bash-templater

This is the project site

0

To expand on @FooF's great answer (newlines didn't format in comment), using a heredoc + control characters, you can permit arbitrary characters and filenames:

template() {
    # if [ "$#" -eq 0 ] ; then return; fi # or just error
    eval "cat <<$(printf '\x04\x04\x04');
$(cat $1)
"
}

This accepts any non-null character, and only truncates early if 3 ^D bytes are encountered on their own line (realistically never). zsh even supports null terminators, so printf '\x00\x00\x00' would work. template even works for treacherous filenames like:

for num in `seq 10`; do
    template 'foo "$ .html' # works
done

Beware shell templates can "expand" arbitrary commands, e.g. $(launch_nukes.sh --target $(curl -sL https://freegeoip.app/csv/ | cut -d, -f 9,10)). With great power…

Edit: if you really don't want your files launching nukes at you, just sudo -u nobody sh (or some other safe user) beforehand.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.