I have a large file of MySQL insert statements that I need to submit to a database.
Normally I would just "cat $FILE|mysql -u$USER -p$PASS database".
The problem is, that the file occasionally has errors in it, causing this command to fail.
It is safe for me to skip the lines with errors, but undesirable to re-submit the same data more than once.
- what is the best way for me to parse this file and submit it to the database?
I have written a bash script which works - except when the file is big it takes way too long (it deletes each line after it processes it so that interrupting the script will not cause data to be re-submitted again, which is a slow process)