Don't forget that an intruder that compromises that server could also use it as a springboard to attack other customers' facilities behind the firewall. Thus, security risk is not solely confined to the first customer's assets.
Get some justification from the vendor as to why they can't use the VPN. If there is genuinely no alternative to RDP connection direct to the server then they need to take responsibility for any security breaches through that connection. Bear in mind that the vendor has just admitted to security flaws in their application architecture by stating that there is something about the application that precludes the use of the VPN.
Make the access conditional on their signing an agreement indemnifying you against any damage caused by a security breach through the RDP connection. In addition you should require them to obtain suitable professional indemnity or liability cover or provide proof of existing cover with terms that would cover this situation.
In short, make the vendor prove that they can afford to pay for any damages and make their access conditional on a contractual obligation to do so.