Questions tagged [awk]
The AWK utility is a data extraction and reporting tool that uses a data-driven scripting language consisting of a set of actions to be taken against textual data (either in files or data streams) for the purpose of producing formatted reports.
195
questions
0
votes
1
answer
92
views
Bash script : Print all lines between two pattern in variables with slashes
I have a directory that looks like so :
/path/to/files/data-file.1
/path/to/files/data-file.2
/path/to/files/data-file.3
/path/to/files/data-file.4
/path/to/files/data-file.5
/path/to/files/data-file....
2
votes
2
answers
228
views
Best way to filter output by date range
I need to filter a row of a csv file by a date.
The file is structured as so:
[email protected] active 01/24/11 10:04 07/23/23 16:56
...
0
votes
1
answer
165
views
How to save input colors in awk?
I've seen many more or less related topics, but all I found discuss how to color awk output. I need to preserve the color that was provided as awk input. For example, now I have:
./some_command
(...
2
votes
1
answer
432
views
Possible to parse ldapsearch output?
I have an ldapsearch query to pull some data of an email system, I am trying to parse the data into something I can analyze in a table/flat file, and generate reports on the accounts.
Sample of the ...
2
votes
2
answers
233
views
Convert lscpu output to row based format in excel/csv table format
I have the below data (lscpu output) for 1000 servers and I want the same in excel but in row format and not as in command output in columns
1.2.3.4 CHANGED
Architecture: ...
-1
votes
1
answer
99
views
Copy/Move Entire Line To End Of The Line Above
I have a file that looks like the following:
/path/to/a/very/long/path
0 0 0 0 0 0
I need to move the line starting with a number (this can be any value not necessarily 0) to the end of the ...
4
votes
1
answer
1k
views
How to correctly escape single quotes in AWK
It isn't clear exactly what more needs to be escaped in the following macro to allow it to be used with awk or sed on FreeBSD.
define(`RELAY_MAILER_ARGS', `TCP $h 2525')dnl
Here is an awk command ...
0
votes
1
answer
354
views
awk to process multiple lines and return several strings
I have these strings:
Certificate Name: domain1.com
Serial Number: 37aadb8fae7b908e1228c3e71
Domains: domain1.com www.domain1.com
Expiry Date: 2022-11-20 02:14:29+00:00 (VALID: 3 days)
...
0
votes
0
answers
67
views
GAWK not running in crontab
I have the below script running successfully manually. But when it is executed in crontab (below), I dont see anything happening.
Script
#!/usr/bin/gawk -f
#!/bin/bash
PATH=/usr/local/bin:/usr/bin
#...
0
votes
1
answer
122
views
Variable in AWK
How can I add a variable within awk command? below is my command:
cat /home/ubuntu/test/copy.txt | awk 'NR==21{print ".host = "$line";"}1'
$line is basically an IP address which ...
0
votes
1
answer
107
views
PHP Serialize AWK command speed up?
I've built an SQL migration script that will replace text in a MySQL dump file. Replacements would be easy with sed, but we must account for special serializations (PHP Serialize). The code below ...
1
vote
1
answer
301
views
how to exclude folders by date with find?
This is "myfolder":
tree -a 'myfolder'
myfolder
├── 20220902
│ ├── filefoo
│ └── filebar
├── 20221001
│ ├── filefoo
│ └── filebar
└── 20221015
├── filefoo
├── filebar
└── ...
1
vote
1
answer
224
views
awk pattern search with file input to search in another file
I am looking for some hints, how I could achive a better, more elegant, efficient solution.
So I've got a file containing a list of names.
names.txt:
Doe John
del Super Mario
van Hoppity Jenny
van der ...
0
votes
1
answer
583
views
how can I extract a Version number from a website with curl and grep / sed?
I want to get the latets version number from this website:
https://www.mailpiler.org/wiki/download
so I do a curl to get the content and then just use a pipe to search for the string using grep
curl ...
1
vote
1
answer
350
views
awk $1 and $2 variables coming up empty
GOAL : compare the size of our directory structure day over day. The data folder has over 990tb in it, so I had to run a bunch of parallel du's in order for it to finish in a reasonable amount of ...
0
votes
1
answer
468
views
Using awk with column number in a bash script
I'm using a command like this to get some connections from nf_conntrack:
awk '($3 == "tcp") && ($6 != "TIME_WAIT") && ($10 == "dport=1234")' /proc/net/...
1
vote
0
answers
1k
views
Replace a string from the cat output
I need to replace a string with the string which is mentioned in a file abcd.txt in all files in a folder.
For eg: I want to replace a string 'apple' with whatever the cat output of this file abcd.txt
...
0
votes
1
answer
627
views
php-fpm & Apache 2 - analysing PHP Message: logs
One of my servers has recently been switched to using php-fpm.
The error logs now log 404's in a new format:
[Sun Dec 26 00:11:37.827426 2021] [proxy_fcgi:error] [pid 25239:tid
140600822003456] [...
0
votes
1
answer
42
views
Can Linux do conditional judgment to insert the content at a certain line in Linux
Can Linux do conditional judgment to insert the content at a certain line in Linux?
For example, I would like to add user "test" to /etc/sudoers to let it can switch to root:
1 #
2 # ...
0
votes
1
answer
173
views
sed: customizing config file header with a defined length?
I use sed to customize LXC container configuration files from the LXC host. That works well so far. When adjusting comment headers (hostname and date), there are aesthetic problems with the width of ...
0
votes
2
answers
391
views
Awk script to filter text output to single line
I have a script that is giving me an output like this:
ruby-devel is needed by software1
tcl is needed by software 2
python3 is needed by software 3
ocaml is needed by software 1
I'm new to awk but ...
2
votes
1
answer
567
views
Extract a ids from text file
I have a text file live below
{
"data": {
"assigned-entity-id": null,
"assigned-entity-type": null,
"availability-domain": null,
"...
0
votes
1
answer
328
views
Finding a string and remove all contents between two delimiters
I have searched and I don't know what I'm doing wrong but I can't find an answer to this question.
I have a file with all text stored as a single line. I am needing to find a pattern and remove all ...
1
vote
1
answer
110
views
Misaligned AWK output
I have simple awk command generating output and then sending an email:
awk '(NR==FNR){a[$2]=sprintf("%.2f",$1*value); next} {print $1,$2,a[$2]}' OFS="\t\t\t" value=$COST /tmp/1.txt ...
1
vote
1
answer
146
views
Format export using grep and awk
I would like to export data from zimbra server, but it's export data in every new lines, my command:
zmprov -l gaa -v domain.com | grep -e "zimbraForeignPrincipal: " -e "...
0
votes
0
answers
157
views
AWK to read multiple files in linux without duplicates
I have a task to merge three files in linux, so i used awk to achieve this task.
hostsfiles is a variable which has test.txt test1.txt and test3.txt
awk 1 {{ hostsfiles | join(' ') }}
I get below ...
0
votes
1
answer
59
views
Multiply first column by variable depending on unit
I have du -sh output saved to txt file:
2.1G /home/x/sample/2GB/
101M /home/x/sample/100MB/
...
What I want is to multiply each value in the first column by a value taken from variable ...
0
votes
1
answer
115
views
Find all words after specific string
So, here's the thing.
I have a file contains like this:
LOG=123 HEY=BRO FOO=BAR LOG=124
I need the value of LOG= and an output like this:
123
124
I already tried using this command:
echo "$LOG&...
0
votes
1
answer
1k
views
How to remove text after a 4 digit year in a file name
I am looking to remove all text after the 4 digit year of movie files:
Input:
Some.Movie.Name.2011.1080p.BluRay.x265.mp4
Another.Movie.Name.1999.1080p.BluRay.x264.mp4
Another.Movie.Name.II.2001.1080p....
0
votes
1
answer
182
views
left outer join 2 shell output on a common key column using bash
Is there a way I can do a "left outer join" for bash output from 2 different commands based on a common key column (ip address)? For various reasons, I would like to avoid saving the output ...
0
votes
1
answer
386
views
How to change one line of code in a block of code from linux command line?
I have a change I need to make across multiple different sites and may need to make again in the future, so I'd like to put it in a shell script. Essentially I want to find a specific class block in a ...
0
votes
2
answers
343
views
nethogs -t | awk '{print $0}' return no output
nethogs is a linux process bandwidth monitoring tool. https://github.com/raboof/nethogs
nethogs -t
it's output looks like this.
Refreshing:
sshd: root@pts/0/4002/0 0.185156 0.0773437
unknown ...
0
votes
1
answer
2k
views
How to separate variables using awk and append it to Text file using ansible
What I got as extra variable to ansible playbook is "CHOW_app/timmy_app1/johnn_app3/harper_app4/mona_app5". This is passed as single variable to the playbook. I have to separate it and save it in a ...
0
votes
1
answer
61
views
How to grep lines only with matched words in Linux [closed]
I have a little question.
I have a list of processes.
After this command
ps aux | grep postgres
I see
postgres 1178 0.0 0.2 320064 27060 ? S Apr12 0:05 /usr/lib/postgresql/10/bin/...
0
votes
2
answers
5k
views
uptime | awk -F: '{print $NF}' - What happens here? [closed]
Can someone explain me what happens here?
uptime | awk -F: '{print $NF}'
-F: field separator
$NF: number of fields in the current line
I don't understand why this command result in the same result ...
0
votes
2
answers
729
views
Extract string from column with grep
I have a column based .txt file and I want to grep output the Common Name, having a little trouble.
V 300223164711Z 01 unknown /C=UK/O=LMG/OU=server/CN=server/name=server/emailAddress=...
0
votes
1
answer
169
views
How to print only Users list and Access level from /etc/sudoers?
We have multiple servers and i would like to print users list and access level using bash.
I tried below command but could not get exact result what i wanted to:
awk '/Allow root/{p=5} p > 0 {...
0
votes
2
answers
2k
views
How do I cut just the file name from from ls -l output? [closed]
I am in the process of concatenating ogg audio files in bulk with the help of ffmpeg using a mylist.txt file
The format of the mylist.txt file is
file '/path/to/file1.wav'
file '/path/to/file2.wav'
...
0
votes
1
answer
179
views
Convert date and time from output
I have a file with date and time on a couple thousand lines.
I need to convert:
2019/08/02-01:23:50+0000
To:
02/Aug/2019-01:23:50 +0000
I so far have
th=`grep ...
0
votes
1
answer
246
views
Convert date format in file
I have a file with the current date format
11.22.33.44 - - [2019/08/02-01:23:50+0000] GET www.
1111:2222:3333:4444:5555:6666:7777:8888 - - [2019/08/02-01:28:18+0000] GET www.
And i need to convert ...
3
votes
3
answers
3k
views
CentOS 7 how to get interface name by macaddress?
I'm trying to figure out how I can get the network interface name and then change the name. Right now it's easy to change the name with this example:
ip link set eth1 name <newName>;
However I ...
8
votes
2
answers
26k
views
How to make in SSH private key from one line, three lines [closed]
i have ssh-key, something like this
-----BEGIN RSA PRIVATE KEY----- my_super_secret_password -----END RSA PRIVATE KEY-----
Of course this key does not work. When i am doing manual things, something, ...
3
votes
2
answers
2k
views
How to add the filename as prefix to each line of the text file? (batch processing; subdirectories)
I have many .txt files with their filenames being different dates:
01.01.2010.txt
02.01.2010.txt
...
Currently each file (e.g.: 01.01.2010.txt) looks like this:
0.351 XXX XXX XXX XXX
0.292 ...
0
votes
2
answers
132
views
sed/awk - find match by one string and change different string on the same line
this should be easy question for most of you but for me.
I need to perform some text operations above json a yml config files, which contains definitions of some channels - structure of every line is ...
0
votes
2
answers
782
views
Can't redirect awk output to a file... tried everything
I created a nice little command to measure data bandwidth consumption over a metered connection (excluding local traffic):
sudo iftop -i enp1s0 -f 'not (src net (10 or 172.16/12 or 192.168/16) and ...
1
vote
3
answers
104
views
text processing linux using grep awk or anything else
I want to replace in this string Wed Apr 10 06:44:10 UTC 2019 all whitespaces with comma and along with that trim off UTC part.
What I have tried :
var1="Wed Apr 10 06:44:10 UTC 2019"
echo ${var// /,...
1
vote
1
answer
385
views
get a specific line from in linux using grep
Hello I am new to linux and grep and awk commands.
I am trying to get the number value from this string variable and then store it in another variable.
var1="Version must be incremented on update. ...
0
votes
0
answers
293
views
chkrootkit awk not found
I am using chkrootkit 0.53 on my ubuntu dekstop 18.04
When I was doing scan with `
./chkrootkit -p /folder
it give me error
chkrootkit: can't find 'awk' .
Could anyone help?
Thank you
0
votes
1
answer
480
views
nmap output in csv format sorted by last octet in ip address
I have this command which works well to give me a list of ip address, MAC address, and Mac Vendor
sudo nmap -sn 192.168.0.0/24 | awk '/Nmap scan report for/{printf $5;}/MAC Address:/{print " "substr($...
0
votes
2
answers
322
views
Extract text between 2 strings only if another string is present
I have a quit large XML file that I am trying to modify only certain lines of the file. The lines I want to modify will all contain in them. On those lines, I want to have only the text between 2 ...