Questions tagged [awk]

The AWK utility is a data extraction and reporting tool that uses a data-driven scripting language consisting of a set of actions to be taken against textual data (either in files or data streams) for the purpose of producing formatted reports.

Filter by
Sorted by
Tagged with
0
votes
0answers
23 views

Cat file, get every nth line starting at line number [migrated]

I have a large file. How can I print every 9th line starting at line 6? awk NR % 9 == 0' file1 > file2
0
votes
1answer
122 views

Convert date and time from output

I have a file with date and time on a couple thousand lines. I need to convert: 2019/08/02-01:23:50+0000 To: 02/Aug/2019-01:23:50 +0000 I so far have th=`grep ...
0
votes
1answer
57 views

Convert date format in file

I have a file with the current date format 11.22.33.44 - - [2019/08/02-01:23:50+0000] GET www. 1111:2222:3333:4444:5555:6666:7777:8888 - - [2019/08/02-01:28:18+0000] GET www. And i need to convert ...
2
votes
3answers
64 views

CentOS 7 how to get interface name by macaddress?

I'm trying to figure out how I can get the network interface name and then change the name. Right now it's easy to change the name with this example: ip link set eth1 name <newName>; However I ...
0
votes
2answers
197 views

How to make in SSH private key from one line, three lines [closed]

i have ssh-key, something like this -----BEGIN RSA PRIVATE KEY----- my_super_secret_password -----END RSA PRIVATE KEY----- Of course this key does not work. When i am doing manual things, something, ...
2
votes
2answers
125 views

How to add the filename as prefix to each line of the text file? (batch processing; subdirectories)

I have many .txt files with their filenames being different dates: 01.01.2010.txt 02.01.2010.txt ... Currently each file (e.g.: 01.01.2010.txt) looks like this: 0.351 XXX XXX XXX XXX 0.292 ...
0
votes
2answers
35 views

sed/awk - find match by one string and change different string on the same line

this should be easy question for most of you but for me. I need to perform some text operations above json a yml config files, which contains definitions of some channels - structure of every line is ...
0
votes
2answers
28 views

Can't redirect awk output to a file… tried everything

I created a nice little command to measure data bandwidth consumption over a metered connection (excluding local traffic): sudo iftop -i enp1s0 -f 'not (src net (10 or 172.16/12 or 192.168/16) and ...
1
vote
3answers
39 views

text processing linux using grep awk or anything else

I want to replace in this string Wed Apr 10 06:44:10 UTC 2019 all whitespaces with comma and along with that trim off UTC part. What I have tried : var1="Wed Apr 10 06:44:10 UTC 2019" echo ${var// /,...
0
votes
1answer
37 views

get a specific line from in linux using grep

Hello I am new to linux and grep and awk commands. I am trying to get the number value from this string variable and then store it in another variable. var1="Version must be incremented on update. ...
0
votes
0answers
60 views

chkrootkit awk not found

I am using chkrootkit 0.53 on my ubuntu dekstop 18.04 When I was doing scan with ` ./chkrootkit -p /folder it give me error chkrootkit: can't find 'awk' . Could anyone help? Thank you
0
votes
1answer
72 views

nmap output in csv format sorted by last octet in ip address

I have this command which works well to give me a list of ip address, MAC address, and Mac Vendor sudo nmap -sn 192.168.0.0/24 | awk '/Nmap scan report for/{printf $5;}/MAC Address:/{print " "substr($...
0
votes
2answers
28 views

Extract text between 2 strings only if another string is present

I have a quit large XML file that I am trying to modify only certain lines of the file. The lines I want to modify will all contain in them. On those lines, I want to have only the text between 2 ...
-3
votes
1answer
49 views

searching file for last matching line, then put 3rd word of this line into a variable

i have a logfile, a script checks this log-file every hour it shall pack the 3rd word of the last matching line into a variable but i didn't manage it to work yet .. logfile: 2018-11-18 22:27:10 ...
2
votes
2answers
93 views

AWK help filter data into CSV

We are doing some security research for which I need to extract from Debian repos all available package names, versions, desc, etc. I'm trying to parse output of apt-cache dumpavail into CSV and ...
0
votes
1answer
59 views

Using sed to remove "(double-quotes) on code lines while leaving them on real comments in a vimrc [closed]

I'm finally learning how to use sed to edit configs while installing Ubuntu(Server/Desktop). After much gnashing of teeth and searching about for the proper syntax to not remove the "(comment double-...
1
vote
1answer
241 views

regex: match two /'s back from end of line [duplicate]

I have a file of file paths. The depth of the directories is of various lengths and path names. I'd like to match two directories backwards (two /'s) and delete the match, creating a new file with the ...
0
votes
1answer
32 views

delete some char set from the last column linux

I want to the delete everything after hyphen "e-" in the last column output.. Below is what my output looks like.. # awk '/out: OpenSSL/ { gsub( /[][]|out:/ , "" ) ; print $1,$2,$3 }' sslcheck.log |...
0
votes
0answers
321 views

sed + delete duplicate lines except the first one according to specific request

in /etc/services file I can see duplicate lines for example the following line is duplicate ( or more ) odasrv 2432/udp so it can easily deleted by sed as the follwing sed -i '/codasrv ...
1
vote
1answer
601 views

How can I use an iperf output and only grep Interval and Bandwidth?

I am trying to get the output from the command iperf -c 10.0.0.1 -t 3600 -i 2 And only need the Interval and bandwidth fields listed for the entire hour of logging. I haven't used grep or awk in ...
-1
votes
1answer
320 views

How to pass variable into bash script and use seperate variables with awk?

I have a script that I am using to clear out all lines of a log files older than X days. X can be passed in via command line as well as the log file name, ex usage: ./purge-catalina.sh 3 /opt/...
1
vote
3answers
662 views

Grep last match from file

I'm running the following command against a log file and only wanting to get the last/latest match. Sometimes there may only be one match, other times there may be multiple, which is causing an issue ...
6
votes
5answers
4k views

Calculate sum of several sizes of files in Bash

I have list of files in a file, cache_temp. In file cache_temp: /home/maildir/mydomain.com/een/new/1491397868.M395935P76076.nm1.mydomain.com,S=1740,W=1777 /home/maildir/mydomain.com/een/new/...
4
votes
2answers
462 views

How to show specific luns from lsscsi command?

I need to grep specific luns from the lsscsi command. For example: [root@e15l1 ~]# lsscsi 4 0 1 | awk '{print $1,$6}' | head [4:0:1:0] - [4:0:1:1] /dev/sdab [4:0:1:2] /dev/sdj [4:0:1:3] /dev/sdz [4:...
1
vote
2answers
72 views

How to extract the last two (or any number) directories from a path with sed?

I'm really looking for a generic solution here, but for the sake of example, let's say I'm looking for the deepest 2 directories in a path. e.g.: Given a path /home/someone/foo/bar, how can I use sed (...
0
votes
2answers
475 views

replacing space with new line characters for vagrant box use

I am trying to setup vagrant box using shell script where in I am storing key and cert files. Normal format of any cert or key file is as below -----BEGIN CERTIFICATE----- ...
-1
votes
1answer
95 views

validate file content with bash Regular Expressions + Linux

how to validate the following file content that should be include single integer/float number by bash Regular Expression or any other idea with awk/sed cat /var/VERSION/Version_F35_project_usa 2....
0
votes
2answers
362 views

How to insert a complex line between two single quotes in sed?

I have this line in the file /var/lib/pgsql/data/postgresql.conf: # archive_command = '' # command to use to archive a logfile segment we need to uncomment the archive_command parameter, ...
1
vote
1answer
207 views

multiline into single line and output to logger

I have a log file where data entries are as follows Each entry starts with time: time: 20170509123420 dn: uid=abc,ou=People,dc=test,dc=example,dc=com changetype: modify replace: ...
1
vote
2answers
172 views

How to retrieve first N and last M lines from input in fish?

I have an input like: 1 2 3 4 5 6 7 8 And I'd like to retrieve only N first and M last lines from this input. For example, if N is 2 and M is 4 I want the result to be: 1 2 5 6 7 8 Lines can ...
-1
votes
1answer
81 views

how do i remove local ip from list of ip addresses?

I have a variable that contains list of ips and i will like to remove the ip of the instance where the variable is set Please see below [root@ip-10-10-2-100 ~]# INSTANCE_IP=$(curl -s http://169.254....
9
votes
2answers
14k views

iptables show just one chain

tldr: How can I get iptables to show just one chain? I can have iptables show just one table, but a table consists of multiple chains. I need to find where in chain INPUT is the last rule (usually ...
0
votes
0answers
114 views

Nagios php script to read iostat values

I'm working on a php script(nagios plugin) to read r/s,w/s,rMB/s,wMB/s,%util from the iostat command, e.g. #iostat -xm sda avg-cpu: %user %nice %system %iowait %steal %idle 1.05 0....
0
votes
1answer
62 views

Parsing the time data from last command which has total “00:00” time spent on the last column

I have last command output for the hundred of Servers collected into the File,the problem is now is output is "Feb,Jan,Feb" meaning that its taking the Feb output of last years as well, So, i'm ...
0
votes
1answer
112 views

list services and store them in array bash

I want to list all services from /etc/init.d/ and grep pattern and store them in array. for instance like below. ls -la /etc/init.d/ | grep example-subs | awk '{ print $9 }' example-subs-sidekick ...
0
votes
2answers
93 views

Replace a string once another string has been found

I'm programmatically building boxes, but I need to update a .repo file for PHP which has the following contents: # Repository: http://rpms.remirepo.net/ # Blog: http://blog.remirepo.net/ # ...
0
votes
1answer
2k views

Date format conversion to epoch using gawk ( awk )

I want to convert dates in a file to epoch time. A sample csv looks like this: $ cat test.jtl 2016/09/19 00:31:17.080,238,test1 2016/09/19 00:31:17.154,322,test2 22-Sep-2014 10:32:35.012,222,...
2
votes
2answers
3k views

Pipe grep result to awk with “: ” as field separator

I would like to filter the result of a grep command, eg : myRepo/path/to/my/file.php:123: error_log(' - myError'); If I do the following, it works. echo " myRepo/...
1
vote
1answer
119 views

How to write a shell script to search from a position in file instead to start from beginning of the file again

I am trying to write a shell script to find value stored after a repetitive word in a big file with millions of lines using for loop. After the finding the repetitive word how i make script to start ...
0
votes
1answer
710 views

How to combine lsof stat and ps to get more info in one line bash command

I can use lsof to get top open files stat for processes, as below: $ lsof -n|awk '{print $2}'|sort|uniq -c|sort -nr | head -n 5 17955 11945 10282 2786 5980 32152 1920 27803 1786 32107 ...
0
votes
3answers
207 views

Matching first block of text from a file, between 2 strings

I'm trying to match only the first block of text between 2 strings: 'menuentry ' and '}' from a file as follows: Input file: (some irrelevant text) menuentry 'My Customized Linux, with Linux 3.10.0-...
-1
votes
1answer
83 views

How to use the awk,sed Modify the text?

As follows, some text 16G 16G 1.9G 40G 4G 4G 952M 60G 16G 16G 1.6G 40G 5G 780M 5G 80G I want to change all the unit from 'M' to 'G', like this 16G 16G ...
0
votes
2answers
589 views

Prepend String using perl/awk/sed?

I'm trying to script a change to my apache config file (httpd.conf). I'm trying to match the following string: # # DirectoryIndex: sets the file that Apache will serve if a directory And PREPEND the ...
0
votes
2answers
658 views

Shell Script to check CPU Utilisation

I am trying to write a shell script which will check free memory of the server. I am trying below code top -M -n1 | grep "Mem" | awk '{print 0 + $7}' | awk '{ print $1 / 1024 }' | awk '{if ($1 < ...
0
votes
1answer
72 views

Limit awk results to html requests

I'm using awk to analyze some access log files. I'm currently using the following: awk '($9 ~ /404/)' access_log | awk '{print $9,$7}' | sort | uniq -c | sort > 404.txt Which returns all of the ...
2
votes
2answers
1k views

AWK + Host lookup

Currently using the following line to get top 20 ips sorted by requests: grep 'GET /' /var/log/nginx/access.log | awk '{ print $1 }' | sort -n | uniq -c | sort -rn | head -20 Output: 575 66.249.*.* ...
1
vote
1answer
778 views

Running CAT in SCREEN no output to file

I am trying to code this command to run in screen along with 2 others that will be using cat to read a file then pipe it into awk for filtering. Here is the command; screen -d -m /bin/cat /var/www/...
1
vote
2answers
2k views

Special char or wildcard in awk field separator

Would appreciate to know about the special char or wildcard in the following command: /sbin/ifconfig eth0 | awk -F ' *|:' '/inet addr/{print $4}' What does the code below mean? awk -F ' *|:' '
2
votes
1answer
299 views

how to get the desired value in bash shell from the lshw output?

I want to extract the info from lshw output where it says: -cpu product: ...... I want only the product info of this -cpu. How can I do that with awk or sed?
0
votes
2answers
476 views

Print last column containing spaces and special characters

I need a full path/command thats been execued to start a process thats exposed on the network - listens on tcp/udp . I did manage to get all of that using below "one liner": netstat -nlp46|tail -n +3 ...