Compare two user lists in bash

Question

I have two different user lists. I need to compare the users and make sure they exist in both files. One is the passwd file, and another is a flat file for that has the usernames and other information that I can extract a sorted username list from.

This gives me a sorted list of the usernames:

cat /etc/passwd | cut --fields=1 | sort -k1.2

Is there a better way to do this number one, and number two how do I then compare it the other user list from the other file? If the user does not exist I will be adding it to the flat file.

Answer 1

This should work for you using process substitution with bash, diff, awk, and sort:

diff <(awk -F: '{print $1}' /etc/passwd | sort) <(sort your_other_list_file)

This assumes your your_other_list_file only contains usernames, one per line. Can't help you parse that unless you post an example line.

Answer 2

comm -3 <( (cut -f1 -d: /etc/passwd | sort) ) <(sort file2)

I like this best, it's a one-liner, avoids awk, and comm gives you options to give you entries unique to list1, list2, or commonalities to both. Also cutting before sorting gives sort shorter strings to work with so it might be faster on longer lists.

Answer 3

Let's say your flat file looks like this example:

data info username number status

Then you could do this:

join -v 1 -1 1 -2 3 <(sed 's/:/ /g' /etc/passwd | sort) <(sort your_file) >> your_file

That appends lines that appear in file one (-v 1) that don't appear in file two to the end of file two. The fields for matching are field 1 for file one and field 3 for file two. The sed command compensates for the password file being colon-delimited and the flat file being space delimited. You can adjust field numbers and delimiters to suit your needs.

Answer 4

You can easily see which lines are not present from one file to another with grep:

fgrep -vxf file1 file2

It will show lines present in file2 but not in file1. (Note the order of the arguments)

Answer 5

How about using diff? Just execute diff file_a file_b and you'll get a diff file with all the changes. After that you can simply filter on a + to get the missing lines.

Do note that for diff to work properly you'll probably need to sort it first.

Answer 6

cat /etc/passwd1 | cut -d':' -f1 | sort > passwd_file1
cat /etc/passwd2 | cut -d':' -f1 | sort > passwd_file2
diff passwd_file1 passwd_file2

You could also just sort it and diff it without cutting to get full information.