6

How do I remove lines of a file (hosts) which contain "adf.ly" string?

Maythux
  • 84,289
wb9688
  • 1,437

6 Answers6

13

using sed

Run this command:

sed -i '/adf\.ly/d' inputfile

man sed

   -i[SUFFIX], --in-place[=SUFFIX]
      edit files in place (makes backup if extension supplied)

Using grep

Thanks for @kos notes:

grep -v "ad\.fly" inputFile  > outputfile
Maythux
  • 84,289
  • 2
    You're doing two different things in the sed and in the grep version: in the sed version you're deleting all the lines containing an ad.fly substring while in the grep version you're deleting only lines containing an ad.fly substring surrounded by a non-word character (i.e. not surrounded by [A-Za-z0-9_]); in the grep command also the . matches any character (i.e. it will exclude lines containing e.g. adxfly) and you're not using any of the EREs' syntax in the pattern, so the -E option is not really needed. – kos Jun 16 '15 at 09:11
  • However I didn't downvote this, drop me a comment if you ever correct this and I'll upvote it since the sed version is fine – kos Jun 16 '15 at 09:35
  • @kos Even you downvote it's legitimated :). I removed the grep for now – Maythux Jun 16 '15 at 09:35
  • Didn't saw your comment, anyway why did you remove the grep version? Just fix it, it can be useful – kos Jun 16 '15 at 09:38
  • 1
    I will fix it before adding it again – Maythux Jun 16 '15 at 09:39
  • Ok, upvoted the sed version then – kos Jun 16 '15 at 09:47
  • No, the point is the -w option forces grep to match only ad.fly substring not surrounded by an alphanumerical or _ character, so you should drop it if you want to make the grep version act the same as your sed version, which doesn't cater for that (and I guess that's the way it should act, since it looks like OP is trying to skip lines containing links). Also the pattern is wrong, the . in particular means "any character", i.e. lines containing strings such as adxfly, adyfly and so on will be matched. Just use the same pattern used in the sed version: grep -v "ad\.fly" – kos Jun 16 '15 at 09:54
  • It works with w but I dont take care of _ and other special characters, though i fixed it again. thanks – Maythux Jun 16 '15 at 09:57
  • You could use grep -Fv adf.ly input.txt > output.txt for better performance on very large files. – evilsoup Jun 16 '15 at 17:45
  • 1
    +1 for grep, which is IMO the most straightforward way to do this. – Gaurav Jun 16 '15 at 18:33
4

The following will remove the lines containing "adf.ly" in filename.txt in-place:

sed -i '/adf\.ly/d' filename.txt

Use the above command without -i to test it before removing lines.

muru
  • 197,895
  • 55
  • 485
  • 740
Ron
  • 20,638
3

Using awk (thanks to terdon for the shortened version):

< inputfile awk '!/adf\.ly/' > outputfile
  • < inputfile: redirects the content of inputfile to awk's stdin
  • > outputfile: redirects the content of awk's stdout to outputfile

awk command breakdown:

  • !/adf\.ly/: prints the record if not matching the adf\.ly regex

Using Perl (thanks to terdon for the shortened version):

< inputfile perl -ne '/adf\.ly/||print' > outputfile
  • -n: places a while (<>) {[...]} loop around the script
  • -e: reads the script from the arguments

Perl command breakdown:

  • /: starts the pattern
  • adf\.ly: matches an adf\.ly string
  • /stops the pattern
  • ||: executes the following command only if the pattern didn't match the line
  • print: prints the line
kos
  • 35,891
  • 1
    Why not just awk '!/ad\.fly/' in > out? You could also shorten the Perl to perl -ne '/ad\.fly/||print' file if you're into golfing :) – terdon Jun 16 '15 at 12:03
  • @terdon Because I'm silly (awk) and anyway not that proficient in both languages. Thanks for your suggestions (by the way I didn't get the golfing reference!) – kos Jun 16 '15 at 12:19
  • 1
    Code golfing is the game of making code as concise as possible, usually at the price of sacrificing legibility. My point is that the perl alternative I gave is indeed shorter but yours was much clearer and more understandable to a non-expert. – terdon Jun 16 '15 at 13:11
  • @terdon Ah I see, yes I'd say I am then. After all it's sill explained right after – kos Jun 16 '15 at 13:29
1

Here is a bash one-liner:

while IFS= read -r i; do [[ ! $i =~ .*adf\.ly.* ]] && echo "$i"; done <file.txt

  • Variable i will contain each line while iterating

  • [[ $i =~ .*adf\.ly.* ]] checks if the line has the string adf.ly, ! negates the check so [[ ! $i =~ .*adf\.ly.* ]] will check if the line does not contain adf.ly

  • If yes (&&), then the line will be printed.

To save the output to another file (out.txt):

while IFS= read -r i; do [[ ! $i =~ .*adf\.ly.* ]] && echo "$i"; done <file.txt >out.txt
heemayl
  • 91,753
1

You could use the standard text editor, ed:

printf '%s\n' 'g/adf\.ly/d' w q | ed file.txt
evilsoup
  • 4,485
1

You can use Vim in Ex mode:

ex -sc 'g/adf\.ly/d' -cx hosts

  1. g global search

  2. d delete

  3. x save and close

Zombo
  • 1