0

I have a process that tails our AWS CloudWatch logs. It works great, except for the fact that every line is prepended by the CloudWatch group and container ID. For example:

core-api 04a7ce3daf83aa018a92eb4a613c87354695dc8219f213697f7f786b81d4d [PROD] - INFO: GET 200 - 5ms core-api 04a7ce3daf83aa018a92eb4a613c87354695dc8219f213697f7f786b81d4d [PROD] - INFO: POST 200 - 7ms

Is there a way to pipe my logs command I use with some type of filter to only show the line info after [PROD] -?

4

You can try using cut to cut out the columns you are interested in.

tail -f whatever | cut -d ' ' -f 3-

[PROD] - INFO: GET 200 - 5ms
[PROD] - INFO: POST 200 - 7ms

This -d sets a space as the field delimiter, and -f specifies to display only the third and subsequent fields.

Specifying fields can get more complex, too. Suppose you only really want to get rid of that annoying useless second field. You can then specify -f 1,3-.

core-api [PROD] - INFO: GET 200 - 5ms
core-api [PROD] - INFO: POST 200 - 7ms

See the cut man page for more things you can do with this command.

1

You could try using awk. If the lines are all prepended by 4 fields then

 ...| awk '{$1=$2=$3=$4=""; print $0}'

but you may need to explicitly flush the output

  ...| awk '{$1=$2=$3=$4=""; print $0; fflush() }'

This basically nulls the first 4 fields. Note that if you then wanted to print just certain fields the first field is still $5, but then I guess if you wanted to do that you wouldn't need to null the fields in the first place.

If there were a variable number of fields and they always terminated with [PROD] - then you could use sed

...| sed s '/^.* \[PROD\] -//'

which basically substitutes everything from the start of the line to the - with null //.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.