How can I use bash to replace a string in a file from another file

Question

How can I use bash to replace a string in a file from another file For example:

There is a path: \\Hostname\example\example.txt I want to replace the Hostname with 162 hostnames from another file that contains the 162 hostnames then I save the result. So the total path should be : 162 paths with different hostnames (162)

The first folder is : Paths.txt ( has 2 paths):

\\Hostname\example1\example1.txt
\\Hostname\example2\example2.txt

The Second file is: Hostname.txt ( has 162 hostnames )

I tried :

sed -i 's/Hostname/Hostname.txt/gI' Paths.txt

But the result is :

\\Hostname.txt\example1\example1.txt
\\Hostname.txt\example2\example2.txt

It did not read the real hostname inside the file Hostname.txt

Answer 1

This is not a straight answer to the question, but to solve this task you can use the following loop:

while IFS= read -r hostname; do \
    printf '\\\\%s\\example1\\example1.txt\n' "$hostname" >> ./hosts-new.txt; \
    printf '\\\\%s\\example2\\example2.txt\n' "$hostname" >> ./hosts-new.txt; \
done < ./hosts-list.txt

Copy the above lines and run them as a single command (inline script).

Here are some explanations:

• hosts-new.txt is the output file.

• hosts-list.txt is the input file containing hostnames, each at new line.

• IFS= read -r hostname (at each loop's iteration) will read a line of the input file and assign it as a value of the variable that we named $hostname... as @StéphaneChazelas says in his encyclopedic answer this is the canonical way to read one line of input with the read builtin.

  • The key is that read reads words from a (possibly backslash-continued) line, where words are $IFS delimited and backslash can be used to escape the delimiters (or continue lines). So the read command should be tweaked to read lines.

  • IFS= changes the internal field separator to the null string, thus we preserve leading and trailing whitespace in the result.

  • The option -r - raw input - disables interpretation of backslash escapes and line-continuation in the read data (reference).

• printf '\\\\%s\\example\\example.txt\n' "$hostname" will generate the desired output for each line. By \\ we are escaping the special meaning of the single \. The character \n at the end means new line. %s is a string formatted placeholder of the value of the first positional parameter (of printf) which is our variable $hostname.

• The output redirection >> will append each line to the output file.

• By the part < ./hosts-list.txt at the end we are passing the content of the input file as input of the while loop, which is defined by the three keywords while (condition); do (something); done. So while reads the file line by line and, in this case, the condition is while there is an input.

---

Here is another version of the above, that probably will be faster because it will write to the new file only once after the loop is finished.

while IFS= read -r hostname; do \
    printf '\\\\%s\\example1\\example1.txt\n' "$hostname"; \
    printf '\\\\%s\\example2\\example2.txt\n' "$hostname"; \
done < ./hosts-list.txt > ./hosts-new.txt

Note here is used the redirection > (instead of >>) that will override the content of the output file.

Answer 2

If you really want to read both the input paths and replacements from files, you could do something like this in GNU awk:

gawk '
  BEGIN {
    PROCINFO["sorted_in"] = "@ind_num_asc"
  }
  NR==FNR {
    paths[FNR] = $0
    next
  }
  {
    for(i in paths)
      print gensub(/\\\\Hostname/,"\\\\"$0,"1",paths[i])
  }
' Paths.txt Hostname.txt

You can omit the BEGIN block if you don't care about the order of the paths for each host. You can probably safely omit the \\\\ and \\\\ in the gensub pattern and replacement (they're just included to anchor the Hostname string, in case it may may occur without the \\ prefix elsewhere in the file).

You could also consider slurping the whole content of the Paths.txt file into a single multi-line string, and applying gensub with the g global flag instead of the loop:

gawk '
  NR==FNR {
    paths = paths (paths ? RS : "") $0
    next
  } 
  {
    print gensub(/\\\\Hostname/,"\\\\"$0,"g",paths)
  }
' Paths.txt Hostname.txt

or slurping the paths file as a single null-delimited record in a BEGIN block

gawk -v pathsfile="Paths.txt" '
  BEGIN {
    RS = "\0"
    getline paths < pathsfile
    RS = "\n"
  }
  {
    printf "%s", gensub(/\\\\Hostname/,"\\\\"$0,"g",paths)
  }
' Hostname.txt

or (should work in any awk)

awk '
  NR==FNR {
    paths = paths (paths ? RS : "") $0
    next
  } 
  {
    newpaths = paths
    gsub(/\\\\Hostname/,"\\\\"$0,newpaths)
    print newpaths
  }
' Paths.txt Hostname.txt