Tagged Questions

The AWK utility is a data extraction and reporting tool that uses a data-driven scripting language consisting of a set of actions to be taken against textual data (either in files or data streams) for the purpose of producing formatted reports.

learn more… | top users | synonyms

1
vote
1answer
22 views

How to substitute a string from an awk result?

I have a JSON file which contain multiple occurrence of replaceme. I only want to substitute an occurrence based on a specific string found a few line above. E.g., [...] "title" : { ...
0
votes
1answer
27 views

Pipe lsof to grep to awk ps -ef to grep

I'm trying to get some info on a certain PID. I'm probably doing this in a manner thats way too difficult, but I'm trying to run this: lsof | grep grepstuff | awk '{ps -ef | grep $2;}' I'm not ...
0
votes
3answers
36 views

Modify config file settings from bash [closed]

I need to modify a config file from a bash script, changing the values of particular settings. I need to make the change using common Linux tools (sed/awk/tr/etc.). The config file can have ...
0
votes
2answers
23 views

Replace date in epoch (seconds) format to normal in a log file

I am on a ubuntu machine trying to read from log files and found the logs contain date and time in epoch format(seconds). How can I convert these dates in numbers into any readable format from command ...
0
votes
2answers
86 views

How to print regex match

My problme is two fold. First is printing a regex match. The second is the actual regex. Problem 1: I'm trying to print the output of yum grouplist --verbose into a format that I can put into a ...
3
votes
3answers
521 views

Year Month day from file name in shell script

I hava file names like below adn_DF9D_20140515_0001.log adn_DF9D_20140515_0002.log adn_DF9D_20140515_0003.log adn_DF9D_20140515_0004.log adn_DF9D_20140515_0005.log adn_DF9D_20140515_0006.log ...
0
votes
1answer
82 views

Eliminate need for grep use only awk

I am trying to generate a list of PID's to pass to strace I only want the PID's php-fpm. I can get the correct output by using grep with awk but am looking for an awk only answer. Here is what I ...
1
vote
1answer
285 views

Get some meaning from apache logs with awk and grep [duplicate]

I am trying to have some meaning from my apache log files: I want to parse my access log and have some statistics about 200 status code hits (how many times each of them was hit). So I tried to learn ...
1
vote
3answers
145 views

drop ips with iptables from awk result [closed]

I have a log file like this : Frq:15 IP:0.0.0.0 Date: 2014-03-21 12:19:31 AM Frq:9 IP:198.252.206.25 Date: 2014-03-21 12:19:31 AM Now i get column of ip addresses with awk : cat /tmp/test | awk ...
2
votes
3answers
104 views

awk ignores the record separator I set for the first record

I am trying to change the record separator but awk does not use the record separator I set until the second record and beyond. I get output like this: ARRAY /dev/md4 level=raid1 num-devices=2 ...
0
votes
2answers
358 views

Script for search string inside the file for every one hour

I need to write a script where the particular string(alphanumeric) should be searched inside the file with every hour for whole day. And the search should not contain the previous hour results. Time ...
0
votes
1answer
68 views

Invalidating email addresses from sql dump before import

Synopsis I would like to import production data into the development environment so testing can be done in an almost accurate way. Of course I need to invalidate e-mail addresses as I don't want ...
0
votes
3answers
65 views

Tinydns data file change ip with bash

I am using tinydns and need to dynamicly change some ip in data file. I want to use bash script for it. For example data file: +sub1.example.org:282.196.222.245:14400 ...
0
votes
1answer
32 views

Parse requests per OS type apache log

I'm trying to return a sorted list of requests per OS type, parsed from an Apache access_log file (not combined format as I need the user-agent info). Here's an example of the output I'm looking for: ...
-1
votes
2answers
125 views

How to find DEVICE on which read per second (rd_sec/s) is highest?

I am using awk to find out DEVICE on which read per second (rd_sec/s) is highest, (from SAR file) below is my command : # sar -d -f sa28 | awk '{print $3,$4}' |sort -k1,1n Output is showing on ...
2
votes
2answers
1k views

Puppet & Facter - how to determine if running Cent 6 or Cent 5

How would you use facter and puppet to determine if the OS is running Cent 6.x or Cent 5.x ? facter operatingsystemrelease 6.4 I only care about the major release (6) I've thought about using ...
1
vote
1answer
73 views

Search log files newer than a time

I have a log file which has entries like below. I want to find only the log entries newer than a given time. What would be a suitable shell script for it? [2013-06-23 21:31:19,972] INFO [2013-06-23 ...
0
votes
1answer
92 views

Changing Zone files with sed

I have this dns zone file I was unable to change the resource record type A to CNAME @ IN SOA ns1.domain.com. webdev.domain.com. ( 2006080101 ; serial 8H ; refresh 2H ...
1
vote
2answers
178 views

How to use AWK to generate bandwidth report in Apache

I have my Apache logs set up like this: LogFormat "%v %t %I %O" billing How can I use AWK to generate a report which shows me the total bandwidth (received + sent) in MB per virtual host? Here's ...
0
votes
1answer
93 views

Finding out total PHP usage in MB's

I'm trying to find out total memory used by all PHP processes on my CentOS server. After some grepping and awking this is my final command. ps -e | grep php | awk '{print $1}' | xargs pmap | grep ...
1
vote
2answers
348 views

Yum command to downgrade packages

I need to downgrade php on a server and found the command below. Something seems to be wrong with the syntax. yum downgrade $(rpm -qa --qf "%{NAME} " | \ grep -v ioncube | grep ^php | ...
0
votes
3answers
415 views

tail-pipe-awk script, fine on Centos but weird on Ubuntu

I wrote a script on CentOS and now I'm trying to run it on Ubuntu and it is behaving unexpectedly. Here's something like the script, which I run under bash: sudo tail -F ...
1
vote
1answer
223 views

Logging varnishlog output to a file with awk in daemon mode

I would like to daemonize the answer in this topic http://serverfault.com/a/480871/94127. Since varnishlog has daemon mode, I tried to use -D option as below. But the timestamps are not appearing. ...
0
votes
4answers
202 views

Move files to subdirectories: /img/ab123.jpg --> /img/ab/ab123.jpg

I have a linux webserver with about 25.000 images in one directory. They are all lowercase and in the format two letters, three digits and jpg as the extension. Because that many files in one folder ...
0
votes
1answer
637 views

Need equivalent of command with nmap, awk and grep for Windows machines with PowerShell and/or Perl

Please see this thread for reference How can I scan using nmap and Zenmap all hostnames that begin with a particular string? One of the answers in the thread above uses the following query (I take ...
4
votes
3answers
419 views

Determine number of unique lines with awk or similar in bash

I am using AWK to read through a custom log file I have. The format is something like this: [12:08:00 +0000] 192.168.2.3 98374 "CONNECT 192.168.2.4:8091 HTTP/1.0" 200 Right now, I have AWK (from ...
0
votes
0answers
174 views

How to get range time a log file in shell script

How to get range time using awk command? I have two variables $from and $to with format (hh:mm:ss) I want to get the range time between two variables. For example: get all range time between: ...
-1
votes
2answers
477 views

linux/solaris insert date & time at the beginning of each line in file

I want to add date & time at the begging of each line in some file I used sed in order to add the date & time before each line in file please advice what I need to update in my sed in order ...
0
votes
1answer
227 views

Regex matching specific details if first match exists, multiline

I'm struggling to get the correct regex to match specifically 'Contact' and 'User-Agent' only if the 'Contact' address matches 10.0.x.x in ~70GB of SIP messages. The SIP message will always contain a ...
0
votes
2answers
459 views

remove non-printing characters

I have a tab delimited text file. When I open the file with vi editor, I can see ^M character (in blue color). This is not part of my data. It got there because the original SQL data had carriage ...
0
votes
1answer
148 views

grep + match value only if value in the end of line

please advice what is wrong with my code, ( I run this code on both OS linux and solaris ) I don't get the line from: "file ended with .tmp" example: I need to verify if $FILE ended with .tmp ...
0
votes
3answers
1k views

Remove a sentence which starts with word “mark” and ends with “;”

I am having 100's of files that contain sentence that starts with mark and ends with semicolon(;). eg: mark is driving a car; i need to remove every sentence starts with mark and end with ";" How ...
-3
votes
2answers
314 views

linux + solaris + delete all characters in file except numbers and dot char [closed]

please advice how to delete all characters in file except numbers and "." Should be implemented by sed or awk or perl ( one line ) or any other idea , so I will add the syntax in my ksh script ...
2
votes
3answers
173 views

linux + match only VALID IP from text file into other file

please advice how to match only the valid IPs ( 255.255.255.255 ) from the file.txt and insert only the valid IP into VALID_IP.txt file ( see VALID_IP.txt for example ) the solution should be ...
0
votes
1answer
75 views

Remove all words that DOESN'T contain a colon [closed]

I have a file with this type of lines: blabla bla:bla whatever what:ever alpha bravo charlie del:ta I only want to keep the words that contain a colon. So the output for the above would be: ...
0
votes
3answers
229 views

How to find largest files in a dir - ignoring directory sizes

I want to do something like this du -a | sort -rn | head But I want to extract files only, ignoring directories. To be clear, I want to traverse through all sub-directories but I don't want to ...
1
vote
1answer
392 views

Writing to /etc/networking/interfaces at boot using sed/awk?

Newbie here, I'm trying to write to an auto-generated /etc/network/interfaces file of a newely provisioned XEN Ubuntu (12.04/10.04/8.04) DomU server at boot time using (currently) sed. The ...
3
votes
2answers
233 views

Extracting a line section of mysql backup using sed

I occasionally need to extract a single record from a mysqlbackup To do this, I first extract the single table I want from the backup... sed -n -e '/CREATE TABLE.*usertext/,/CREATE TABLE/p' ...
0
votes
2answers
127 views

linux + create host file from CSV file by sed or awk or perl

I have the following CSV file this file defined which Linux machine exist in the system and there ip's my target is to create host file from this file please advice how to create host file as ...
5
votes
1answer
328 views

File size limit exceeded in bash

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem the script has problem like this: cat file | awk '{print $1" "$2" "$3}' | sort -n > result the file's size ...
2
votes
2answers
1k views

How can I add a line break before a set of values going into a bash script?

I've got a set of data piping into a bash script. Here's an example of what that data looks like: "foo1": "Miscellaneous text", "foo2": "More text", "foo3": "blah blah blah", "foo4": ...
7
votes
2answers
1k views

bash sub-string extraction for specified start and end character

For example I have a log file having this entry: [Wed Aug 08 11:39:41 2012] [error] [client 155.94.70.224] ModSecurity: [file ...
8
votes
2answers
2k views

How to get all fingerprints for .ssh/authorized_keys(2) file

Is there a simple way to get a list of all fingerprints entered in the .ssh/authorized_keys || .ssh/authorized_keys2 file? ssh-keygen -l -f .ssh/authorized_keys will only return fingerprint of ...
0
votes
2answers
211 views

looking for a tricky one liner to get diff of awk output

Lets say i have two files like below: file1 a 23 b 34 c 47 file2 a justin_beiber_sucks b segmentation fault c jackson d nop Now what i want is to get a diff of the first column of two file. I'm ...
3
votes
1answer
333 views

Alternatives to gnuplot+awk?

I've needed to do some more sophisticated graphing and plotting as of late. While I know that gnuplot is up to the task, and can do it all from the command line, are there other, simpler or more ...
0
votes
3answers
252 views

How to join a variable number of lines

I have a file with Feature names and a variable number of usernames below: Feature1 user1 user2 Feature2 user3 I'd like to be able to move the usernames to the same line as the Feature name ...
5
votes
3answers
13k views

How to delete all characters in one line after “]” with sed?

How to delete all characters in one line after "]" with sed ? Im trying to grep some file using cat, awk. Now my oneliner returns me something like 121.122.121.111] other characters in logs from ...
2
votes
1answer
103 views

Memory usage REGEX

Could you help me to modify the following one-liner to get more precise memory usage - right now it's just an integer I would like to include first value after coma. free -m | awk '/Mem:/ {tot = $2;} ...
2
votes
2answers
154 views

Gathering IP's from a complicated log

I have a question regarding the use of some more advanced grep, awk, sed. I have a log file, for a proprietary MTA, that contains IP's in a string, delimited by [redacted]^~x.x.x.x^[redacted]. So far ...
1
vote
2answers
81 views

linux or solaris match files with uniq syntax

how to match all files that start with the string OS_PATCH and ended with .gz remark - the solution must to fit Linux and Solaris OS this task could be by find command or maybe with ls with ...