Yes this will work, but it is a bad idea. Sysprep maintains a count of how many times it's been run. If you do this, you'll max out that count and make a lot of work for yourself.
Install all of your drivers on the first machine, then sysprep it.
If you're using WDS, you can push drivers with the image. Also, you can use dism to add drivers to an offline image before you push it down.
EDIT
Since you're running XP, there is a Microsoft approved way of adding drivers to the image. This might work out much better for you. Check out this KB article:
http://support.microsoft.com/kb/314479