Concepts / Executing External Commands from Python

Executing External Commands from Python

The central idea is to let Python prepare the command while the external zip utility performs the archiving.

  • Programming

From Manual Backups to Automation

Backing up files manually is tedious and error-prone. You might forget a directory, overwrite yesterday's backup, or use an inconsistent naming scheme. A Python backup program automates the process: it selects the files or directories, creates an archive whose name records when the backup was made, and runs the backup command with one execution.

Key point

The central idea is to let Python prepare the command while the external zip utility performs the archiving.

The Execution Journey

The program has a clear execution journey. Python first selects source directories and chooses the target directory. It then formats the current date and time into a timestamp, joins that timestamp to the target directory, and creates one command string. os.system() passes that string to the operating system shell. The shell runs zip, and Python receives a return value after the external command finishes.

Python programconstructs commandos.system()passes commandOperating systemshellruns commandzip utilitycreates archiveReturn value0 or non-zerocommand stringexecuterun zipcompletion statusstatus returned
What happens after Python starts constructing and executing the backup command?

The diagram separates responsibilities. Python does not perform the compression itself in this program. It handles file selection, naming, and command construction. The zip utility handles the actual recursive archiving. After zip finishes, Python uses the returned status to decide which message to print.

Sources and targetdirectories and archivepathTimestampformatted date and timeCommand stringzip -r target sourcesZip archivetimestamped backupStatus code0 or non-zeroBackup messagesuccess or failurecombine pathsname archiveexecutefinishcheck result
How does the command string move from Python into the operating system, and how does the result return to Python?

Building the Backup Command

The command is assembled in stages. First, the target filename is formed from the backup directory, a directory separator, a timestamp, and the .zip extension. Next, the source list is converted into one space-separated string. Finally, the program combines the zip utility, the recursive option, the target filename, and the source string into one command.

ziparchive utility-rrecursive operationtarget.ziparchive destinationsource directoriesspace-separated listfollowed bythenthen
How are the source directories, timestamped archive filename, and zip option combined into one executable command?
Code walkthrough
import os
import time

source = ["/home/user/docs", "/home/user/photos"]
target_dir = "/home/user/backups"

timestamp = time.strftime("%Y%m%d%H%M%S")
target = target_dir + os.sep + timestamp + ".zip"
source_list = " ".join(source)
command = "zip -r {0} {1}".format(target, source_list)

print(command)
result = os.system(command)

if result == 0:
    print("Backup completed successfully")
else:
    print("Backup failed")
  1. 1
    Import os for os.sep and os.system(). Import time for time.strftime().
  2. 2
    Store the source directories and the directory where the archive should be created.
  3. 3
    Format the current date and time as year, month, day, hour, minute, and second.
  4. 4
    Construct the archive path without hardcoding a separator.
  5. 5
    Convert the list of source directories into one space-separated string.
  6. 6
    Build the complete zip command using the target and source strings.
  7. 7
    Print the command so the user can see what is about to execute.
  8. 8
    Run the command and save its return value.
  9. 9
    Print a success message when the return value is 0; otherwise print a failure message.
  • The source list contains the directories to include in the backup.
  • The target directory is combined with os.sep, the timestamp, and the .zip extension.
  • The format string places the target filename at position 0 and the joined source list at position 1.
  • os.system() executes the constructed command and returns a status value.
  • The final condition treats 0 as success and a non-zero value as failure.
Output
zip -r /home/user/backups/20240115143022.zip /home/user/docs /home/user/photos
Backup completed successfully

In this example, the timestamped archive path is the command's target. The two source directories follow it as a space-separated list. The -r option tells zip to work recursively through subdirectories. The exact timestamp changes with the current date and time, so the displayed archive name is an illustrative concrete value.

Portable Archive Paths

A path is made from directory names and separators. The separator differs by operating system: Windows uses a backslash, while macOS and Linux use a forward slash. os.sep supplies the appropriate separator for the operating system where the program is running. Using it avoids hardcoding one platform's separator into the target path.

Windowsbackup\archive.zipmacOS and Linuxbackup/archive.zipArchive filenametimestamp.zipos.sepchosen automaticallybackslashforward slashsame archive purposesame archive purpose
What changes in the constructed path across operating systems, and what remains the same?
Key point

The archive's purpose and timestamp format stay the same across platforms; os.sep supplies the platform-specific directory separator used when assembling the target path.

Timestamped Archive Names

time.strftime() converts the current date and time into a formatted string. The format string %Y%m%d%H%M%S uses codes for the year, month, day, hour, minute, and second. For example, a timestamp such as 20240115143022 represents a date and time in year-month-day-hour-minute-second order. Adding .zip produces an archive filename that records when the backup was created.

Worked example

Constructing One Archive Name

Assume the formatted time is 20240115143022 and the target directory is /home/user/backups. What target path is assembled?

  1. 1
    Format the time

    time.strftime() produces the timestamp string 20240115143022 from the format codes %Y%m%d%H%M%S.

  2. 2
    Add the separator

    The program places os.sep between the target directory and the timestamp.

  3. 3
    Add the extension

    The program appends .zip to identify the archive filename.

Result

/home/user/backups/20240115143022.zip on a system using the forward slash separator

The timestamp is important because it gives each backup a name based on when it was created. This avoids an inconsistent naming scheme and helps preserve older archive names instead of treating every backup as the same file.

Following the Shell Boundary

The Python statement os.system(command) executes the command as if it had been typed into a terminal or command prompt. The command string crosses from Python to the operating system shell. The shell then invokes zip with the options, target, and source directories contained in that string.

Command executionos.system(command)0success messageNon-zerofailure messagecommand succeedscommand fails
What value does Python receive after the external command finishes, and how does that value guide the result message?

A return value of 0 indicates that the command succeeded. A non-zero error code indicates failure. The program does not need to guess whether the archive command worked; it checks the value and reports success or failure accordingly.

What do you think happens?

The command finishes and os.system() returns 0. Which branch should print its message?

  • The success branch

  • The failure branch

Reveal answer
Answer:

The success branch

os.system() returns 0 when the command succeeds. The program checks result == 0 before printing the success message.

A Concrete Execution Trace

Assume a Linux system with source directories /home/user/docs and /home/user/photos, a target directory of /home/user/backups, and a formatted timestamp of 20240115143022. The program follows a predictable sequence from values in Python to the final status check.

  1. 1
    Python stores the two source paths in source.
  2. 2
    time.strftime() produces the timestamp string 20240115143022.
  3. 3
    The target path becomes /home/user/backups/20240115143022.zip.
  4. 4
    join() turns the source list into /home/user/docs /home/user/photos.
  5. 5
    format() inserts the target and source strings into zip -r {target} {sources}.
  6. 6
    print() displays the command before it is executed.
  7. 7
    os.system() passes the command to the shell, which runs zip.
  8. 8
    Python receives 0 for success or a non-zero error code for failure and prints the corresponding message.

The resulting archive is related to the original source directories as a collected backup: the source directories are supplied to zip, and the timestamped .zip path is the archive destination. The -r option causes zip to work through subdirectories.

Mistakes Beginners Make

  • Common mistake

    Hardcoding a directory separator into the constructed target path.

    What it looks like

    Combining the target directory and timestamp with a separator chosen for only one operating system.

    Why it fails

    Windows uses a backslash, while macOS and Linux use a forward slash.

    Do this instead

    Use os.sep when joining the target directory, timestamp, and .zip extension.

  • Common mistake

    Passing the source list directly into the command instead of joining it.

    What it looks like

    Using the list object where the command needs one string of source paths.

    Why it fails

    The program constructs the zip command from a space-separated source string.

    Do this instead

    Use " ".join(source) before inserting the sources into the command.

  • Common mistake

    Treating every return value from os.system() as success.

    What it looks like

    Printing a success message without checking the result returned by os.system().

    Why it fails

    os.system() returns 0 for success and a non-zero error code for failure.

    Do this instead

    Check whether result == 0 and use the other branch for a non-zero result.

  • Common mistake

    Running the example without customizing source and target_dir.

    What it looks like

    Leaving the example directories unchanged when they do not match the local system.

    Why it fails

    The command must refer to the files and target directory intended for the current system.

    Do this instead

    Set source and target_dir to the appropriate paths before running the program.

Practice the Execution Trace

MEDIUM

Suppose source contains two directories, target_dir identifies a backup directory, and time.strftime() returns a timestamp. Trace the values through these stages: construct target, join source, build command, execute with os.system(), and choose the success or failure message from the returned value.

Hints
  • The target combines target_dir, os.sep, the timestamp, and .zip.
  • The source list becomes one string with spaces between its items.
  • The command contains zip, -r, the target, and the joined source string.
  • A return value of 0 selects success; a non-zero value selects failure.
  1. 1
    Choose source directories and a target directory appropriate for your operating system.
  2. 2
    Use time.strftime("%Y%m%d%H%M%S") to create the timestamp.
  3. 3
    Build the target archive path with os.sep.
  4. 4
    Join the source list with a space.
  5. 5
    Construct the zip command and print it.
  6. 6
    Run os.system(command).
  7. 7
    Check the returned value and report success for 0 or failure for a non-zero value.

Automation Principles

This backup program demonstrates separation of concerns. Python handles file selection, timestamped naming, path construction, command construction, and result checking. The external zip utility handles compression and recursive archiving. The program is portable because it uses os.sep and standard time format codes rather than hardcoding system-specific details, and it is verifiable because it checks os.system() instead of assuming that the command succeeded.

  1. 1
    A Python backup program can construct and execute a zip command to automate file archiving.
  2. 2
    os.sep supplies the operating system's directory separator when the target archive path is assembled.
  3. 3
    time.strftime() creates a timestamped archive name from format codes such as %Y, %m, and %d.
  4. 4
    The source list is joined into a space-separated string before it is inserted into the command.
  5. 5
    os.system() passes the command to the shell and returns 0 for success or a non-zero error code for failure.

Key Takeaways

  • Python can automate backups by constructing a timestamped zip command and passing it to the operating system.
  • Using os.sep instead of a hardcoded separator makes target path construction portable across Windows, macOS, and Linux.
  • time.strftime() supplies the timestamp used in the archive filename.
  • The source list must be joined into one space-separated string before command construction.
  • os.system() returns 0 for success and a non-zero error code for failure, allowing the program to report the execution result.

Related Topics

Related concepts

More concepts starting with E