Our server are hosting a website, but we find our database are added some tables, so we suspect if the server was hacked ?
How can I confirm this and solve it ?
The easiest first step is to use rootkit finding applications, you don't say which operating system you're using, but on Linux this would be chkrootkit or rkhunter. These should tell you if any of your binary files have been altered for malicious versions.
Generally, don't use any of the binaries on the system itself to look for hacking, as if they are malicious they likely will exclude output of exactly what you're looking for.
Next up, tools like unhide that help you find hidden processes and TCP/UDP ports.
After that, I'd do a vulnerability scan with something like openVAS to find out what holes you have in the system.
What you do after that depends on what you find, but in short if you find anything suspicious, nuke it and reinstall is my preferred option. Otherwise you're fighting on their terms.
If you truly suspect the server is hacked, you are best off setting up a new server, ensuring it is hardened for web hosting, restoring your data from a backup, and replacing the existing one with this.
Knowing whether or not your server truly has been hacked may take some time, during which you absolutely want to have the server offline and isolated from the rest of your network.
You say it is the presence of some extra database tables that makes you think the server has been hacked. What do these tables relate to? Web content? I guess you are using IIS (version?) and some database. You won't get much more help unless you provide more specific details.
Are you encoding and decoding your input from your website? Are you using stored procedures etc???
Make sure they not "hacking" your system via the supplied user interface.
Look at the time stamp when the tables were created/modified and then look for similar entries and IPs in your web logs.. i.e. basic cross checking. If there is corresponding entries then try to determine which are legitimate entries and which may be the profile of an attacker. If there are no corresponding entries then they may have either been scrubbed or the website interface was not involved in the attack. i.e. server was otherwise compromised.
Then automate whatever checks you made and regularly check your system.