How to embed a binary file in a bash shell script

Last updated on February 25, 2021 by Dan Nanni

Have you ever been in a situation where you want to include a binary file inside your shell script before sharing it with others? Why would you want to do that? For example, you are a sysadmin in an IT department of your company, supporting Linux users. You want the users to be able to install the company's VPN software and configure everything necessary on their work computer simply by running a shell script you provide. In this case, you create a tarball that contains everything (e.g., VPN software to install, an XML profile and config files to be placed in /etc), and ship the tar file as embedded in an installation script. In another scenario, you are writing a shell script that relies on an external executable for certain tasks. For portability of the script, you want to embed the executable itself within the script. Whether or not embedding a binary blob in a shell script is recommended, the ability to do so may come in handy in certain situations.

In this tutorial, let's find out how you can include a binary file in a bash script and how to retrieve it from the script.

Embed a Tar File in a Shell Script

The way you can embed a binary file such as a tarball in a shell script is by appending the content of the file at the end of the script. Read on to find out how to embed a tar file in a shell script.

First, prepare a shell script (script.sh) as follows. What the script does is to extract an embedded tar file and perform some actions (to be defined in process_tar()) with the extracted content. So fill out process_tar() based on your requirement.

#!/bin/bash

# define the tasks that need to be done with the extracted content
process_tar() {
    cd $WORK_DIR
    # do something with the extracted content
}

# line number where payload starts
PAYLOAD_LINE=$(awk '/^__PAYLOAD_BEGINS__/ { print NR + 1; exit 0; }' $0)

# directory where a tarball is to be extracted
WORK_DIR=/tmp

# extract the embedded tar file
tail -n +${PAYLOAD_LINE} $0 | tar -zpvx -C $WORK_DIR

# perform actions with the extracted content
process_tar

exit 0
__PAYLOAD_BEGINS__

Be careful that __PAYLOAD_BEGINS__ should be the last line of the script. Even empty lines or newline characters are not allowed beyond this line.

Once the shell script is finalized and ready, the next step is to append an actual tar file (dummy.tgz) to the script by running:

$ cat dummy.tgz >> script.sh

At this point, the script is ready to be distributed. When invoked from the command line, the script will extract the embedded tarball in WORK_DIR directory, and process the extracted content as defined in process_tar().

Beware that since the tar file is appended to the shell script as binary data, it is not safe to re-edit the shell script (even the text portion of it) after embedding. If you modify and save the shell script after a tarball is appended, you may see the following error when running it.

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Embed a Tar File in Encoded Format in a Shell Script

Instead of embedding a binary file as is inside a shell script, it is often safer to embed the file in an encoded form. A popular binary-to-text encoding scheme is Base64, which is implemented by the base64 command-line utility.

In order to use Base64-based encoding/decoding, you can replace the line in the above script:

tail -n +${PAYLOAD_LINE} $0 | tar -zpvx -C $WORK_DIR

With:

tail -n +${PAYLOAD_LINE} $0 | base64 -d | tar -zpvx -C $WORK_DIR

This will decode the embedded content before untarring it.

Then you need to append a tar file in an encoded format by running:

$ base64 dummy.tgz >> run.sh

At this point, the shell script is ready to be distributed. You will notice that the tar file is encoded in a textual format, and you may re-edit the script.

__PAYLOAD_BEGINS__
H4sIAJ0WOGAAA+y8B1RUydY/2kgSBBGkJQsiGaHpppscJEvOGcktNJkmZwEBJSoKKCoIKEgWSZJF
chSQKCBZsuQM3a8RZ0bHe+fOvPXd7//+b1msTZ3Tlfbev1371Kmzals4OtxAWIEA/83Eg0n8MNhx
DuaH8Xyf/5YAYCgPmI8HzAvm4QfwgCF8YB4AA+y/ytW35ObiaoZkYABYe7k5uMAdbf9dvf9U/n9p
sjjB39LN3t6L+/jmvzDGMcB8UOjfwR/CB4Vg8Ify8fACGHj+C7z8lH7h/wf+LnCkO8IC/j89xj/A
/9v8h/GCf+H/v5KczCxszazg/9UHwD/w/xB+2PH8h/BDeX/5//+N9Bv+Xx2AiTuYG8bHbQk3/x8d
4y/mPxjMA+HlB0N/n/8QCPR4/kPBsF/z/38jMatJy3KBufmJCZmLMYmYEMzAw+BobkNMKCIC0vJy

Embed a Binary Executable in Encoded Format in a Shell Script

If you want to include a binary executable in a shell script, you can use the same technique described here to save the embedded binary as an executable, and invoke it as needed. See the example script (script2.sh) below.

#!/bin/bash

# line number where payload starts
PAYLOAD_LINE=$(awk '/^__PAYLOAD_BEGINS__/ { print NR + 1; exit 0; }' $0)

# directory where a binary executable is to be saved
WORK_DIR=/tmp
# name of an embedded binary executable
EXE_NAME=dummy_executable

# extract the embedded binary executable
tail -n +${PAYLOAD_LINE} $0 | base64 -d | cat > ${WORK_DIR}/${EXE_NAME}
chmod +x ${WORK_DIR}/${EXE_NAME}

# run the executable as needed
${WORK_DIR}/${EXE_NAME}

exit 0
__PAYLOAD_BEGINS__

Then same as before, append the binary executable in an encoded form:

$ base64 dummy_executable >> script2.sh

Summary

This tutorial demonstrates how to embed a binary file in a bash shell script. While this tutorial can be used to cover a variety of use cases involving binary data, if all you need is a self-extracting archive, there is a dedicated tool for that.

If you find this tutorial helpful, I recommend you check out the series of bash shell scripting tutorials provided by Xmodulo.

Support Xmodulo

This website is made possible by minimal ads and your gracious donation via PayPal or credit card

Please note that this article is published by Xmodulo.com under a Creative Commons Attribution-ShareAlike 3.0 Unported License. If you would like to use the whole or any part of this article, you need to cite this web page at Xmodulo.com as the original source.