Or you could use the DOS 'FC' command like so (This shows the output of both files so you will have to scan for the differences):
fc.exe filea.txt fileb.txt > diff.txt
fc is an aliase for the Format-Custom cmdlet so be sure to enter the command as fc.exe. Please note that many DOS utilities don't handle UTF-8 encoding
You can also spawn a CMD process and run fc within it.
start cmd "/c ""fc filea.txt fileb.txt >diff.txt"""
This instruct PowerShell to start a process with the 'cmd' program using the parameters in quotes. In the quotes, is the '/c' cmd option to run the command and terminate. the actual command to run by cmd in the process is "fc filea.txt fileb.txt" redirecting the output to the file diff.txt.
You can use the DOS fc.exe from within powershell.