The AWK utility is a data extraction and reporting tool that uses a data-driven scripting language consisting of a set of actions to be taken against textual data (either in files or data streams) for the purpose of producing formatted reports.

learn more… | top users | synonyms

0
votes
2answers
573 views

remove non-printing characters

I have a tab delimited text file. When I open the file with vi editor, I can see ^M character (in blue color). This is not part of my data. It got there because the original SQL data had carriage ...
0
votes
1answer
164 views

grep + match value only if value in the end of line

please advice what is wrong with my code, ( I run this code on both OS linux and solaris ) I don't get the line from: "file ended with .tmp" example: I need to verify if $FILE ended with .tmp ...
0
votes
3answers
1k views

Remove a sentence which starts with word “mark” and ends with “;”

I am having 100's of files that contain sentence that starts with mark and ends with semicolon(;). eg: mark is driving a car; i need to remove every sentence starts with mark and end with ";" How ...
1
vote
2answers
1k views

Grep to find lines starting at pattern A until pattern B is matched

I have a log that contains bits like this: [2012-04-16 15:16:43,827: DEBUG/PoolWorker-2] {'feed': {}, 'bozo': 1, 'bozo_exception': URLError(error(110, 'Connection timed out'),), 'entries': []} ...
2
votes
3answers
196 views

linux + match only VALID IP from text file into other file

please advice how to match only the valid IPs ( 255.255.255.255 ) from the file.txt and insert only the valid IP into VALID_IP.txt file ( see VALID_IP.txt for example ) the solution should be ...
-3
votes
2answers
385 views

linux + solaris + delete all characters in file except numbers and dot char [closed]

please advice how to delete all characters in file except numbers and "." Should be implemented by sed or awk or perl ( one line ) or any other idea , so I will add the syntax in my ksh script ...
0
votes
1answer
76 views

Remove all words that DOESN'T contain a colon [closed]

I have a file with this type of lines: blabla bla:bla whatever what:ever alpha bravo charlie del:ta I only want to keep the words that contain a colon. So the output for the above would be: ...
0
votes
3answers
294 views

How to find largest files in a dir - ignoring directory sizes

I want to do something like this du -a | sort -rn | head But I want to extract files only, ignoring directories. To be clear, I want to traverse through all sub-directories but I don't want to ...
1
vote
1answer
506 views

Writing to /etc/networking/interfaces at boot using sed/awk?

Newbie here, I'm trying to write to an auto-generated /etc/network/interfaces file of a newely provisioned XEN Ubuntu (12.04/10.04/8.04) DomU server at boot time using (currently) sed. The ...
3
votes
2answers
273 views

Extracting a line section of mysql backup using sed

I occasionally need to extract a single record from a mysqlbackup To do this, I first extract the single table I want from the backup... sed -n -e '/CREATE TABLE.*usertext/,/CREATE TABLE/p' ...
3
votes
5answers
2k views

Is it worth while to learn Awk?

I am decent with bash scripting and I am catching on to regex, and a little sed usage. Is learning awk still worth while with all the alternatives out there. Opinions?
0
votes
2answers
142 views

linux + create host file from CSV file by sed or awk or perl

I have the following CSV file this file defined which Linux machine exist in the system and there ip's my target is to create host file from this file please advice how to create host file as ...
5
votes
1answer
398 views

File size limit exceeded in bash

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem the script has problem like this: cat file | awk '{print $1" "$2" "$3}' | sort -n > result the file's size ...
2
votes
2answers
1k views

How can I add a line break before a set of values going into a bash script?

I've got a set of data piping into a bash script. Here's an example of what that data looks like: "foo1": "Miscellaneous text", "foo2": "More text", "foo3": "blah blah blah", "foo4": ...
7
votes
2answers
1k views

bash sub-string extraction for specified start and end character

For example I have a log file having this entry: [Wed Aug 08 11:39:41 2012] [error] [client 155.94.70.224] ModSecurity: [file ...
0
votes
2answers
233 views

looking for a tricky one liner to get diff of awk output

Lets say i have two files like below: file1 a 23 b 34 c 47 file2 a justin_beiber_sucks b segmentation fault c jackson d nop Now what i want is to get a diff of the first column of two file. I'm ...
0
votes
2answers
2k views

sed/awk: extract one number from string

I have many lines like the following being returned from a bash command, with different sourceNodeIds of varying digit lengths: <NodeAssociation sourceNodeId="33654" [...] sourceNodeEntity="Issue" ...
3
votes
1answer
395 views

Alternatives to gnuplot+awk?

I've needed to do some more sophisticated graphing and plotting as of late. While I know that gnuplot is up to the task, and can do it all from the command line, are there other, simpler or more ...
0
votes
3answers
300 views

How to join a variable number of lines

I have a file with Feature names and a variable number of usernames below: Feature1 user1 user2 Feature2 user3 I'd like to be able to move the usernames to the same line as the Feature name ...
2
votes
2answers
168 views

Gathering IP's from a complicated log

I have a question regarding the use of some more advanced grep, awk, sed. I have a log file, for a proprietary MTA, that contains IP's in a string, delimited by [redacted]^~x.x.x.x^[redacted]. So far ...
7
votes
3answers
20k views

How to delete all characters in one line after “]” with sed?

How to delete all characters in one line after "]" with sed ? Im trying to grep some file using cat, awk. Now my oneliner returns me something like 121.122.121.111] other characters in logs from ...
2
votes
1answer
113 views

Memory usage REGEX

Could you help me to modify the following one-liner to get more precise memory usage - right now it's just an integer I would like to include first value after coma. free -m | awk '/Mem:/ {tot = $2;} ...
2
votes
7answers
12k views

How to show available disks on a Linux system?

If i run fdisk -l I get an output as: Disk /dev/sda: 120.0 GB, 120034123776 bytes 255 heads, 63 sectors/track, 14593 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: ...
1
vote
2answers
83 views

linux or solaris match files with uniq syntax

how to match all files that start with the string OS_PATCH and ended with .gz remark - the solution must to fit Linux and Solaris OS this task could be by find command or maybe with ls with ...
16
votes
7answers
2k views

Can awk be used instead?

I would like to get the number from rating as output from this # nc localhost 9571 language: language:en_ZA.UTF-8 language:en_ZW.UTF-8 session-with-name:Ubuntu Classic (No effects):gnome-session ...
2
votes
3answers
387 views

Shell script to block proftp failled attempt

I want to filter and block failed attempt to access my proftp server. Here is an example line from the /var/log/secure file: Jan 2 18:38:25 server1 proftpd[17847]: spy1.XYZ.com ...
2
votes
2answers
144 views

Display nth instance of grepped pattern

So here's my problem. I have an issue with a .csv file (Current.csv) in that there are commas randomly place throughout the data, so awk-ing the file doesn't give me constant column numbers for a ...
1
vote
2answers
411 views

Using wget and Awk to count similar expressions

I am trying to create a script that uses wget to download a data set and then awk to sort though the file and tell you the most common filter used which is $14 column. So far I have the wget function ...
-3
votes
3answers
804 views

How to pass the output of a command as a variable in a bash script?

I'm trying to pass the output of an awk command as a variable in a bash script, but not having much success so any help is appreciated. homedirectory=$(awk /stephen/ /etc/passwd | awk -F/home/ ...
0
votes
1answer
156 views

cutting relevant portions of a file [closed]

I want to cut some portions of binary file starting from a particular pattern and ending at a particular pattern.How do I do that in linux using bash.Can this be done using sed or awk? I have a file ...
2
votes
2answers
176 views

awk + perl + get two arguments in to awk syntax

please advice what wrong with my awk syntax and how to fix it ( this syntax is in my ksh script ) and I run my script on linux machine my target is to get from the file.txt only the lines that ...
-1
votes
3answers
207 views

linux + generate new file with specific structure from text file what the best option

in my Linux machine I have the file orig-file.txt this file include now 4 fields but they could be less or more ( this file generate by other application ) I need advice - what the best option to ...
4
votes
2answers
1k views

AWK and file names with space in it.

I'm trying to parse files with awk to change their names. Everything went well until im started to do this with files with space in file name. File names are something like 11237_712312955_2012-01-04 ...
1
vote
3answers
487 views

append to end of line sed / awk

This is probably easy and something is not clicking for me and my lack of coffee right now. I have a file with multiple lines that begin with a tab then the word GROUP something { Some of these lines ...
0
votes
3answers
85 views

shell script + match MAJOR and CRITICAL strings from log file

I need to match lines that have MAJOR and CRITICAL strings that comes after the word ERROR:< any integer number > please advice how to do that with one awk or sed command? more HW_Log.txt ...
3
votes
2answers
1k views

Octal representation of file/folder permissions using awk?

We have various of machines here with all sorts of hardware and Operating Systems, most of them do regular tasks with bash that an agent executes. However it came to my attention that some Solaris ...
3
votes
2answers
763 views

How do I use sed or awk to remove text?

I have the following line in a file: Linux Release............5.4.2.0-02 12_12_2011_07:31:23 How do I remove all characters before the first number with sed or awk? I wish to get the ...
0
votes
1answer
315 views

Sanitizing dates using sed or awk on an ldapsearch query

I'd like to pipe the output of an ldapsearch to sanitize the dates returned - I presume sed and/or awk can help with this but would appreciate a bit of help. The fixed length date format in the LDIF ...
2
votes
3answers
1k views

awk on vmstat to get si,so

I am writing a shell script to check the vmstat si and so data at various time intervals vmstat 1 sample output: procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b ...
0
votes
1answer
86 views

Parse value of $DatabasePassword from RT_SiteConfig.pm

I'm writing a Bash shell script to backup our Request Tracker (RT) MySQL database on a daily basis. Instead of hard-coding the MySQL database user's password in the shell script I'd like to parse it ...
1
vote
4answers
423 views

double awk commands in same pipe?

Hello I'm trying to do something like this but it wont work, can you guys tell me help me out? /usr/bin/mysql -B -r -h ******** -u******* -p****** -D***** \ -e'SELECT `username`, `uid`, `gid`, ...
19
votes
3answers
926 views

Best way to get the MAC of eth0?

Is there a more efficient way to retrieve the MAC address of a NIC in Linux? This works: ip link show dev eth0 | awk ' /link\/ether/ { print $2 }' but can it be found via something like: cat ...
0
votes
3answers
1k views

AWK - Is it possible to search for pattern, then sort results based on a wildcard?

I am storing a number of individually serialized PHP arrays to a file. Each line of the file contains one serialized array. For example: a:2:{s:4:"name";s:8:"John Doe";s:3:"age";s:2:"20";} ...
3
votes
5answers
1k views

How to remove a certain symbol for a bash script

I have a bash script where I get the disk usage, for example 60%. How can I remove the % symbol? Using grep or awk?
3
votes
2answers
1k views

Multiline data list (ldif file) manipulation with awk or other tool

I'm trying to manipulate an ldif file with multiple entries. My purpose is to parse this existing ldif file, extracting the "givenName" and "sn" attributes, so to generate a "mail" attribute. I was ...
0
votes
3answers
704 views

Parsing a diff file using grep/awk

I'm trying to parse a standard diff of some sql files to return only the delete sections. I have been using grep with the after context (-A) which almost works (only because I know that delete ...
1
vote
3answers
2k views

using awk in a bash script with if

I'm doing a script in which I need to test a string and based on its result I'll decide if I go further or not. The command below works fine (if the string starts with "Clean" it will print 1, ...
1
vote
1answer
272 views

Using Find, Grep, Awk, or Sed To Rename Server After Cloning

My client tells me they have cloned a VM in VMWare of an Ubuntu Linux server. Now it's my job to get into all the files and find out what still has the old server name of "bishop" and change it to ...
1
vote
4answers
1k views

Bash: delete all but most recent n files with prefix x

I have a directory of backups that looks like this: -rw-r--r-- 1 ftpuser ftponly 5610595 May 27 00:01 alpha-114.tar.asc -rw-r--r-- 1 ftpuser ftponly 50559368 May 27 00:04 beta-211.tar.asc ...
0
votes
1answer
431 views

How can i Move or copy files (or folder) from solaris server to removable disk.(Backup Tape drive)

i need to move or copy some zip(.tar,.gz,.Z) files from solaris server to Backup tape drive and also i want to retrive from tapedrive when ever i want without any loss. kindly expain the procedure. i ...