This is probably bordering on a silly question, but if I recently restored a Full mysql DB dump via cat dump.mysql |mysql dbname to a new instance of mysql in order to migrate; and I want to restore the last incremental dump that contains the newest data since that full dump, can I just import that dump the same way into my db and it will just add to it without overwriting the other data (unless the data was modified in which case it should overwrite)?
Database restores are a fair amount more complicated than standard filesystem backups.
To have the ability to do incremental point-in-time restores, you will have typically needed to set up periodic full backups (mysqldump or the like) in combination with binary logging. To perform an "incremental" restore, you'd restore the last full backup and then re-play the binary logs starting with the log file and log position the last full was taken at.
If you do have binary logs in place, then you can use those to perform the incremental restore. If not, it's unlikely that you'll be able to restore your data into a consistent state.