9.8 Search and Replace Text in a File

Problem

You want to search for text in a file and replace that text with something new.

Solution

To search and replace text in a file, first store the content of the file in a variable, and then store the replaced text back in that file, as shown in Example 9-3.

Example 9-3. Replacing text in a file

PS > $filename = "file.txt"
PS > $match = "source text"
PS > $replacement = "replacement text"
PS >
PS > $content = Get-Content $filename
PS > $content
This is some source text that we want
to replace. One of the things you may need
to be careful about with Source
Text is when it spans multiple lines,
and may have different Source Text
capitalization.
PS >
PS > $content = $content -creplace $match,$replacement
PS > $content
This is some replacement text that we want
to replace. One of the things you may need
to be careful about with Source
Text is when it spans multiple lines,
and may have different Source Text
capitalization.
PS > $content | Set-Content $filename

Discussion

Using PowerShell to search and replace text in a file (or many files!) is one of the best examples of using a tool to automate a repetitive task. What could literally take months by hand can be shortened to a few minutes (or hours, at most).

Note

Notice that the Solution uses the -creplace operator to replace text in a case-sensitive manner. This is almost always what you will want to do, as the replacement text uses the exact capitalization that you provide. If the text you want to replace is capitalized in several different ways (as in the term Source Text from the Solution), then search and replace several times with the different possible capitalizations.

Example 9-3 illustrates what is perhaps the simplest (but actually most common) scenario:

You work with an ASCII text file.
You replace some literal text with a literal text replacement.
You don’t worry that the text match might span multiple lines.
Your text file is relatively small.

If some of those assumptions don’t hold true, then this discussion shows you how to tailor the way you search and replace within this file.

Work with files encoded in Unicode or another (OEM) code page

By default, the Set-Content cmdlet assumes that you want the output file to contain plain ASCII text. If you work with a file in another encoding (for example, Unicode or an OEM code page such as Cyrillic), use the -Encoding parameter of the Out-File cmdlet to specify that:

$content | Out-File -Encoding Unicode $filename
$content | Out-File -Encoding OEM $filename

Replace text using a pattern instead of plain text

Although it’s most common to replace one literal string with another literal string, you might want to replace text according to a pattern in some advanced scenarios. One example might be swapping first name and last name. PowerShell supports this type of replacement through its support of regular expressions in its replacement operator:

PS > $content = Get-Content names.txt
PS > $content
John Doe
Mary Smith
PS > $content -replace '(.*) (.*)','$2, $1'
Doe, John
Smith, Mary

Replace text that spans multiple lines

The Get-Content cmdlet used in the Solution retrieves a list of lines from the file. When you use the -replace operator against this array, it replaces your text in each of those lines individually. If your match spans multiple lines, as shown between lines 3 and 4 in Example 9-3, the -replace operator will be unaware of the match and will not perform the replacement.

If you want to replace text that spans multiple lines, then it becomes necessary to stop treating the input text as a collection of lines. Once you stop treating the input as a collection of lines, it’s also important to use a replacement expression that can ignore line breaks, as shown in Example 9-4.

Example 9-4. Replacing text across multiple lines in a file

$singleLine = Get-Content file.txt -Raw
$content = $singleLine -creplace "(?s)Source(\s*)Text",'Replacement$1Text'

The first and second lines of Example 9-4 read the entire content of the file as a single string. They do this by using the -Raw parameter of the Get-Content cmdlet, since the Get-Content cmdlet by default splits the content of the file into individual lines.

The third line of this solution replaces the text by using a regular expression pattern. The section Source(\s*)Text scans for the word Source, followed optionally by some whitespace, followed by the word Text. Since the whitespace portion of the regular expression has parentheses around it, we want to remember exactly what that whitespace was. By default, regular expressions don’t let newline characters count as whitespace, so the first portion of the regular expression uses the single-line option (?s) to allow newline characters to count as whitespace. The replacement portion of the -replace operator replaces that match with Replacement, followed by the exact whitespace from the match that we captured ($1), followed by Text. For more information, see “Simple Operators”.

Replace text in large files

The approaches used so far store the entire contents of the file in memory as they replace the text in them. Once we’ve made the replacements in memory, we write the updated content back to disk. This works well when replacing text in small, medium, and even moderately large files. For extremely large files (for example, more than several hundred megabytes), using this much memory may burden your system and slow down your script. To solve that problem, you can work on the files line by line, rather than with the entire file at once.

Since you’re working with the file line by line, it will still be in use when you try to write replacement text back into it. You can avoid this problem if you write the replacement text into a temporary file until you’ve finished working with the main file. Once you’ve finished scanning through your file, you can delete it and replace it with the temporary file.

$filename = "file.txt"
$temporaryFile = [System.IO.Path]::GetTempFileName()

$match = "source text"
$replacement = "replacement text"

Get-Content $filename |
    ForEach-Object { $_ -creplace $match,$replacement } |
    Add-Content $temporaryFile

Remove-Item $filename
Move-Item $temporaryFile $filename