The AWK utility is a data extraction and reporting tool that uses a data-driven scripting language consisting of a set of actions to be taken against textual data (either in files or data streams) for the purpose of producing formatted reports.
0
votes
1answer
19 views
Finding out total PHP usage in MB's
I'm trying to find out total memory used by all PHP processes on my CentOS server. After some grepping and awking this is my final command.
ps -e | grep php | awk '{print $1}' | xargs pmap | grep ...
1
vote
2answers
42 views
Yum command to downgrade packages
I need to downgrade php on a server and found the command below. Something seems to be wrong with the syntax.
yum downgrade $(rpm -qa --qf "%{NAME} " | \
grep -v ioncube | grep ^php | ...
0
votes
2answers
61 views
tail-pipe-awk script, fine on Centos but weird on Ubuntu
I wrote a script on CentOS and now I'm trying to run it on Ubuntu and it is behaving unexpectedly. Here's something like the script, which I run under bash:
sudo tail -F ...
-3
votes
1answer
62 views
IP to Country (IPs in file) [closed]
I'm trying to create a simple script to do this:
SOURCE: access.log
Needed data:
- IP
- date
- data
I need to create a file with a list of Country, their data usage and within the last hour.
...
1
vote
1answer
52 views
Logging varnishlog output to a file with awk in daemon mode
I would like to daemonize the answer in this topic http://serverfault.com/a/480871/94127. Since varnishlog has daemon mode, I tried to use -D option as below. But the timestamps are not appearing.
...
0
votes
4answers
64 views
Move files to subdirectories: /img/ab123.jpg --> /img/ab/ab123.jpg
I have a linux webserver with about 25.000 images in one directory. They are all lowercase and in the format two letters, three digits and jpg as the extension. Because that many files in one folder ...
0
votes
1answer
137 views
Need equivalent of command with nmap, awk and grep for Windows machines with PowerShell and/or Perl
Please see this thread for reference
How can I scan using nmap and Zenmap all hostnames that begin with a particular string?
One of the answers in the thread above uses the following query (I take ...
0
votes
0answers
54 views
rsync script to rewrite suffix - BASH, awk, sed, perl?
trying to write up a script to put the suffix back.
heres what I have but can't get it to do anything :(
would like it to be name.date.suffix
rsync -zrlpoDtub --suffix=".`date +%Y%m%d%k%M%S`.~" ...
4
votes
3answers
108 views
Determine number of unique lines with awk or similar in bash
I am using AWK to read through a custom log file I have. The format is something like this:
[12:08:00 +0000] 192.168.2.3 98374 "CONNECT 192.168.2.4:8091 HTTP/1.0" 200
Right now, I have AWK (from ...
0
votes
0answers
86 views
How to get range time a log file in shell script
How to get range time using awk command?
I have two variables $from and $to with format (hh:mm:ss)
I want to get the range time between two variables.
For example: get all range time between:
...
-1
votes
2answers
82 views
linux/solaris insert date & time at the beginning of each line in file
I want to add date & time at the begging of each line in some file
I used sed in order to add the date & time before each line in file
please advice what I need to update in my sed in order ...
0
votes
1answer
85 views
Regex matching specific details if first match exists, multiline
I'm struggling to get the correct regex to match specifically 'Contact' and 'User-Agent' only if the 'Contact' address matches 10.0.x.x in ~70GB of SIP messages.
The SIP message will always contain a ...
0
votes
2answers
73 views
remove non-printing characters
I have a tab delimited text file. When I open the file with vi editor, I can see ^M character (in blue color). This is not part of my data.
It got there because the original SQL data had carriage ...
0
votes
1answer
64 views
grep + match value only if value in the end of line
please advice what is wrong with my code, ( I run this code on both OS linux and solaris )
I don't get the line from:
"file ended with .tmp"
example:
I need to verify if $FILE ended with .tmp ...
0
votes
0answers
63 views
How to convert NCSA Combined logs to W3C?
We have a requirement to submit logs in W3C format(http://www.w3.org/TR/WD-logfile.html) for our streaming server, icecast. Icecast has w3c format on their roadmap but currently only output in NCSA ...
0
votes
3answers
212 views
Remove a sentence which starts with word “mark” and ends with “;”
I am having 100's of files that contain sentence that starts with mark and ends with semicolon(;).
eg: mark is driving a car;
i need to remove every sentence starts with mark and end with ";"
How ...
-3
votes
2answers
99 views
linux + solaris + delete all characters in file except numbers and dot char [closed]
please advice how to delete all characters in file except numbers and "."
Should be implemented by sed or awk or perl ( one line ) or any other idea , so I will add the syntax in my ksh script
...
2
votes
3answers
122 views
linux + match only VALID IP from text file into other file
please advice how to match only the valid IPs ( 255.255.255.255 )
from the file.txt
and insert only the valid IP into VALID_IP.txt file
( see VALID_IP.txt for example )
the solution should be ...
0
votes
1answer
58 views
Remove all words that DOESN'T contain a colon [closed]
I have a file with this type of lines:
blabla bla:bla whatever what:ever
alpha bravo charlie
del:ta
I only want to keep the words that contain a colon.
So the output for the above would be:
...
0
votes
3answers
130 views
How to find largest files in a dir - ignoring directory sizes
I want to do something like this
du -a | sort -rn | head
But I want to extract files only, ignoring directories.
To be clear, I want to traverse through all sub-directories but I don't want to ...
1
vote
1answer
140 views
Writing to /etc/networking/interfaces at boot using sed/awk?
Newbie here,
I'm trying to write to an auto-generated /etc/network/interfaces file of a newely provisioned XEN Ubuntu (12.04/10.04/8.04) DomU server at boot time using (currently) sed.
The ...
3
votes
2answers
119 views
Extracting a line section of mysql backup using sed
I occasionally need to extract a single record from a mysqlbackup
To do this, I first extract the single table I want from the backup...
sed -n -e '/CREATE TABLE.*usertext/,/CREATE TABLE/p' ...
0
votes
2answers
96 views
linux + create host file from CSV file by sed or awk or perl
I have the following CSV file
this file defined which Linux machine exist in the system and there ip's
my target is to create host file from this file
please advice how to create host file as ...
5
votes
1answer
130 views
File size limit exceeded in bash
I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem
the script has problem like this:
cat file | awk '{print $1" "$2" "$3}' | sort -n > result
the file's size ...
2
votes
2answers
526 views
How can I add a line break before a set of values going into a bash script?
I've got a set of data piping into a bash script. Here's an example of what that data looks like:
"foo1": "Miscellaneous text",
"foo2": "More text",
"foo3": "blah blah blah",
"foo4": ...
7
votes
2answers
503 views
bash sub-string extraction for specified start and end character
For example I have a log file having this entry:
[Wed Aug 08 11:39:41 2012] [error] [client 155.94.70.224] ModSecurity: [file ...
0
votes
2answers
117 views
looking for a tricky one liner to get diff of awk output
Lets say i have two files like below:
file1
a 23
b 34
c 47
file2
a justin_beiber_sucks
b segmentation fault
c jackson
d nop
Now what i want is to get a diff of the first column of two file. I'm ...
3
votes
1answer
230 views
Alternatives to gnuplot+awk?
I've needed to do some more sophisticated graphing and plotting as of late. While I know that gnuplot is up to the task, and can do it all from the command line, are there other, simpler or more ...
0
votes
3answers
129 views
How to join a variable number of lines
I have a file with Feature names and a variable number of usernames below:
Feature1
user1
user2
Feature2
user3
I'd like to be able to move the usernames to the same line as the Feature name ...
2
votes
3answers
3k views
How to delete all characters in one line after “]” with sed?
How to delete all characters in one line after "]" with sed ?
Im trying to grep some file using cat, awk. Now my oneliner returns me something like
121.122.121.111] other characters in logs from ...
2
votes
1answer
83 views
Memory usage REGEX
Could you help me to modify the following one-liner to get more precise memory usage - right now it's just an integer I would like to include first value after coma.
free -m | awk '/Mem:/ {tot = $2;} ...
2
votes
2answers
95 views
Gathering IP's from a complicated log
I have a question regarding the use of some more advanced grep, awk, sed. I have a log file, for a proprietary MTA, that contains IP's in a string, delimited by [redacted]^~x.x.x.x^[redacted]. So far ...
1
vote
2answers
67 views
linux or solaris match files with uniq syntax
how to match all files that start with the string OS_PATCH
and ended with .gz
remark - the solution must to fit Linux and Solaris OS
this task could be by find command or maybe with ls with ...
0
votes
1answer
367 views
filter body of incoming email in procmail
I've encountered some problem when configuring my linux machine.
We are configuring fetchmail and procmail in order to receive the email from the external mail server.The problem start when we want ...
5
votes
2answers
620 views
How to split a PEM file
Note : This is not really a question because I already found the answer but since I didn't find it easily here I will post it so that it can benefit others.
Question : How to read a concatenated PEM ...
0
votes
3answers
2k views
remove line break using AWK
I am facing some problem regarding my bash script.
Below is my bash code :
#!/bin/bash
cd /root/Msgs/TESTNEW/new
file=myfile.txt
var1=$(awk '(NR==30){print $2}' $file)
var2=$(awk 'NR>=38 ...
2
votes
2answers
105 views
Display nth instance of grepped pattern
So here's my problem. I have an issue with a .csv file (Current.csv) in that there are commas randomly place throughout the data, so awk-ing the file doesn't give me constant column numbers for a ...
1
vote
2answers
206 views
Using wget and Awk to count similar expressions
I am trying to create a script that uses wget to download a data set and then awk to sort though the file and tell you the most common filter used which is $14 column. So far I have the wget function ...
-3
votes
3answers
475 views
How to pass the output of a command as a variable in a bash script?
I'm trying to pass the output of an awk command as a variable in a bash script, but not having much success so any help is appreciated.
homedirectory=$(awk /stephen/ /etc/passwd | awk -F/home/ ...
2
votes
2answers
126 views
awk + perl + get two arguments in to awk syntax
please advice what wrong with my awk syntax and how to fix it ( this syntax is in my ksh script ) and I run my script on linux machine
my target is to get from the file.txt only the lines that ...
0
votes
1answer
133 views
cutting relevant portions of a file [closed]
I want to cut some portions of binary file starting from a particular pattern and ending at a particular pattern.How do I do that in linux using bash.Can this be done using sed or awk?
I have a file ...
0
votes
2answers
250 views
Grep to find lines starting at pattern A until pattern B is matched
I have a log that contains bits like this:
[2012-04-16 15:16:43,827: DEBUG/PoolWorker-2] {'feed': {}, 'bozo': 1, 'bozo_exception': URLError(error(110, 'Connection timed out'),), 'entries': []}
...
-1
votes
3answers
129 views
linux + generate new file with specific structure from text file what the best option
in my Linux machine I have the file orig-file.txt
this file include now 4 fields but they could be less or more ( this file generate by other application )
I need advice - what the best option to ...
4
votes
2answers
559 views
AWK and file names with space in it.
I'm trying to parse files with awk to change their names. Everything went well until im started to do this with files with space in file name. File names are something like 11237_712312955_2012-01-04 ...
13
votes
6answers
2k views
bash/sed/awk/etc remove every other newline
a bash commands outputs this:
Runtime Name: vmhba2:C0:T3:L14
Group State: active
Runtime Name: vmhba3:C0:T0:L14
Group State: active unoptimized
Runtime Name: vmhba2:C0:T1:L14
Group State: active ...
1
vote
3answers
305 views
append to end of line sed / awk
This is probably easy and something is not clicking for me and my lack of coffee right now.
I have a file with multiple lines that begin with a tab then the word GROUP something {
Some of these lines ...
2
votes
2answers
527 views
What command can I use to search sendmail logs for ALL message details involving a specific recipient.
CENTOS 5.x | Sendmail
Occasionally I need to search through sendmail delivery logs to find out what happened to a missing message. This usually involves two (or more) steps:
STEP 1: Search ...
0
votes
3answers
69 views
shell script + match MAJOR and CRITICAL strings from log file
I need to match lines that have MAJOR and CRITICAL strings
that comes after the word ERROR:< any integer number >
please advice how to do that with one awk or sed command?
more HW_Log.txt
...
0
votes
4answers
1k views
Linux shell command to filter a text file by line length
I have a 30gb disk image of a borked partition (think dd if=/dev/sda1 of=diskimage) that I need to recover some text files from. Data carving tools like foremost only work on files with well defined ...
3
votes
2answers
733 views
Octal representation of file/folder permissions using awk?
We have various of machines here with all sorts of hardware and Operating Systems, most of them do regular tasks with bash that an agent executes.
However it came to my attention that some Solaris ...
