Tag Info

Hot answers tagged

39

You can do pretty much anything with apache log files with awk alone. Apache log files are basically whitespace separated, and you can pretend the quotes don't exist, and access whatever information you are interested in by column number. The only time this breaks down is if you have the combined log format and are interested in user agents, at which point ...


37

can't test right now, but ... | paste - - should do it


32

$ nc localhost 9571 | awk -F: '/rating/ { print $2 }'


29

You can use nmap to run a ping scan. nmap -sP 192.168.254.* Starting Nmap 5.00 ( http://nmap.org ) at 2011-03-09 11:32 GMT Host xyzzy.lan (192.168.254.189) is up (0.00022s latency). MAC Address: 00:0C:29:5B:A5:E0 (VMware) Host plugh.lan (192.168.254.196) is up (0.00014s latency). MAC Address: 00:0C:29:2E:78:F1 (VMware) Host foo.lan (192.168.254.200) is up. ...


25

It's at /sys/class/net/eth0/address (or more precisely /sys/devices/pciXXXX:XX/XXXX/net/eth0/address where the XXX is your PCI bus ID, but this varies between systems). (Incidentally, I found this with find /sys -name eth0 and looking at the files in the directories identified.)


18

One thing I've never seen anyone else do, for reasons that I can't imagine, is to change the Apache log file format to a more easily parseable version with the information that actually matters to you. For example, we never use HTTP basic auth, so we don't need to log those fields. I am interested in how long each request takes to serve, so we'll add that ...


13

Quanta beat me to it, but I'll include a sed variant if you're that way inclined: nc localhost 9571 | sed -ne 's/^rating://p' Ditto what MadHatter said, though. Your current solution is perfectly sound. (Although I'd grep for "^rating:" rather than just the word to ensure you only get the line you want.)


13

awk '{ if (length($0) < 16384) print }' yourfile >your_output_file.txt would print lines shorter than 16 kilobytes, as in your own example. Or if you fancy Perl: perl -nle 'if (length($_) < 16384) { print }' yourfile >your_output_file.txt


12

Use nmap. Important to run it as root so you get the MAC addresses. Example: sudo nmap -sP 192.168.1.0/24 Will scan 192.168.1.1 - 192.168.1.255. Look up CIDR notation on wiki if you're not familiar with this subnet notation. You should be able to get nmap from the repos of any recentish Linux distro, e.g. sudo apt-get install nmap or sudo yum ...


11

The issue here is parsing with ls. Consider to take a look here: Why you shouldn't parse the output of ls. The reason why you shouldn't do it is since UNIX allows almost any character in a filename, including whitespace, newlines, commas, pipe symbols, and pretty much anything else you'd ever try to use as a delimiter except NUL. In its default mode, if ...


10

The different alternatives exist, not because each can do the same thing as the other (although this is true for localized cases, or for full programming languages such as Perl), but rather because they can do different things. sed is not very suitable for processing tabular data, and using pure bash for that is usually more pain than one would like. awk ...


10

Forget about awk and grep. Check out asql. Why write unreadable scripts when you can use sql like syntax for querying the logfile. Eg. asql v0.6 - type 'help' for help. asql> load /home/skx/hg/engaging/logs/access.log Loading: /home/skx/hg/engaging/logs/access.log sasql> select COUNT(id) FROM logs 46 asql> alias hits SELECT COUNT(id) FROM logs ...


10

echo "121.122.121.111] other characters in logs from sendmail...." | sed 's/].*//' So if you have a file full of lines like that you can do sed 's/].*//' filename


9

huh what a crazy script :) lets try like this :) #!/bin/bash cat accounts | while read account do echo -e "\n Account: $account \n"; echo sudo ./backup_maildir $account; done if everything looks fine and sudo line works out of script as expected drop the echo and voila


9

Here's another hack using plain bash without temporary files: while read l; do [[ -n $l && ${l###} = $l ]] && ssh-keygen -l -f /dev/stdin <<<$l; done < .ssh/authorized_keys You can easily make it a function in your .bashrc: function fingerprints() { local file="$1" while read l; do [[ -n $l && ${l###} = $l ]] ...


9

I think this will split the line as you want sed -e 's/\]/\]\n/g' log | sed -e 's/^ *//g' | awk '/^\[/ {print}' First put a newline after each ] then remove any leading spaces and finally print the lines beginning with [. Your input line becomes [Wed Aug 08 11:39:41 2012] [error] [client 155.94.70.224] [line "271"] [id "960020"] [rev "2.2.5"] [msg ...


8

This one command will do what you want: grep -o '\[[^]]*\]' inputfile


7

Use tail -n +X filename to print from the Xth line to end of file.


7

You can also just use the shell: nc localhost 9571 | while IFS=: read key val; do [[ $key = "rating" ]] && echo "$val"; done


7

Oh my god, that's awful. Your script uses bash; I suggest you do this instead: #!/bin/bash for i in *.wav; do mv "${i}" "${i%.wav}.ext"; done See the Bash Guide for more details on parameter expansion.


6

As long as you run this command from a host in the same network segment, nmap will report all of the MAC address for each host is discovers. For example: sudo nmap 192.168.1.0/24 -sP Starting Nmap 4.76 ( http://nmap.org ) at 2011-03-09 06:29 EST Host old.net (192.168.1.1) appears to be up. MAC Address: 00:18:39:C5:A1:DC (Cisco-Linksys) For the MAC ...


6

yes, you can (and should) use (one) awk instead of (two) grep and cut: $ nc localhost 9571 | awk -F: '/^rating:/ { print $2 }' Be sure to match you line as good as you can to avoid ugly bugs. /rating/ works, /^rating/ is better (safer), /^rating:/ is best (in this case).


6

One possibility is: awk 'ORS=NR%2?" ":"\n"' If the line number is evenly divisible by 2, end with a new line, otherwise, end with a space. (Tested on: CentOS 6, GNU Awk 3.1.7) Using sed (see explanation): sed ':a;N;$!ba;s/\nGroup/ Group/g' Further reading: Removing new line characters


6

If you want to use sed, there's no reason to read the whole file into memory. You can merge every other line like this: sed 'N;s/\n/ /' inputfile Use any character you'd like instead of the space. Here's another way using awk: awk '{printf "%s", $0; if (getline) print " " $0; else printf "\n"}' inputfile The if/else handles the case where there is an ...


6

How about cut instead: cat logfile | cut -d "]" -f1


6

You'll be farther ahead by extending fail2ban by dropping in a custom configuration file into its .d config directory. Do as little work as possible!


6

If the file name is always like in your example you can use something like: for x in *.log; do year=${x:9:4}; month=${x:13:2}; day=${x:15:2}; [[ ! -d "$year/$month/$day" ]] && mkdir -p "$year/$month/$day"; done This substring extraction is available in bash, not sure about other shells.


5

Like Ignacio Vazquez-Abrams said, it's probably going to stderr. As well, there's an awk trick that will make your life easier: $NF, which is the last field. Thus, I would do something like this: java -jar jmxclient.jar usr:pass host:port java.lang:type=Threading ThreadCount 2>&1 | awk -F: '{print $NF}'


5

You could do something similar to this. for i in `grep -i "someuser@recipientdomain.com" /var/log/maillog | awk '{print $5}'`; do grep -i $i /var/log/maillog; done This will grep out the line for the user you are looking for, then select the 5th item on the line (seperated by spaces iirc). Then for each message ID in that list, will then grep for the ...


5

This was previously answered on StackOverflow : awk ' split_after == 1 {n++;split_after=0} /-----END CERTIFICATE-----/ {split_after=1} {print > "cert" n ".pem"}' < $file



Only top voted, non community-wiki answers of a minimum length are eligible