I have various bash scripts that run, they scan log files for folks who try and make mischief, spam us and so on.

I have been struggling with this for some days trying to figure this out.

I have a text file with a list of with ip's

I use sed to scan the list and remove ip's such as our ip and other known ips and ip ranges. That get added to this list from people who perhaps make a mistake.

for example

In the fist line I am trying to match all 45.182.32.165 and any ip beginning with 45.

Ideally I would like remove the following 45.0.0.0/8 and or any of the net masks up to /24

sed -i '' '/^45.*.*.*/d'  /directory/blocked_subnets/somelist
sed -i '' '/^50.81.238.*/d'  /directory/blocked_subnets/somelist
sed -i '' '/^50.84..*/d'  /directory/blocked_subnets/somelist

These lines do work bit sometimes not as intended.

I have tried various regex's I have found on the net but they don't seem to work.

I was hoping someone who more experience in this can help me refine this sed -i to work properly.

The '' are because I am doing this on a freebsd machine.

Thank you in advance.

Thank you for your help.

Alas none of the above suggestions would work for me.

After much reading and experimentation

I found had to add the -r (to activate the regex) before the -i and this is the format I used for the regex, which seems to work.

sed -r -i '' '/^120[.]152[.][0-9]{1,3}[.][0-9]{1,3}/d' /path/to/some/file

to remove the ip 120.152.35.192 from the file "file"

I tested the regex on the "The Regedx Coach" it seems valid.

However I would welcome any additional input and suggestions to refine the above.

Regards

  • -r doesn't really "activate" the regex, it's indicating that you're using Extended Regular Expressions (as opposed to Basic Regular Expressions). This mostly affects what has to be escaped: quantifiers in BRE are \{1,3\}; in ERE, it's just {1,3}. The characters with changing escaping behaviour between BRE and ERE are |,{},(), and +. For GNU sed, using -r (or the equivalent -E) or not using it only changes what has to be escaped and what not. – Benjamin W. Nov 5 at 20:19

I think your problem is that the . is working as a wildcard matching any character, instead of that use \. which would match a literal ..

Unfortunately I don't know the differences with FreeBSD, but as this is a somewhat simple regex I imagine it should work. I would use as a generic regex (very simplified) for any IP the following:

'/^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/d'
  • ^: Starts with
  • [0-9]: Any digit
  • +: The preceding match happens 1 or more times
  • \.: A literal dot

So now if I wanted to match what you required I would change each octet to match the network:

sed -i '' '/^45\.[0-9]+\.[0-9]+\.[0-9]+/d' /directory/blocked_subnets/somelist
sed -i '' '/^50\.81\.238\.[0-9]+/d' /directory/blocked_subnets/somelist
sed -i '' '/^50\.84\.[0-9]+\.[0-9]+/d' /directory/blocked_subnets/somelist

Also, consider that this will only work if each line of the list starts with the IP (Not even spaces before the IP). If this is not the case just remove the ^

See: https://www.gnu.org/software/sed/manual/html_node/Regular-Expressions.html

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.