if you will only monitor single port on those switches, it might even work, especially if you won't be pulling data too often.
keep in mind that scaling up can be non-trivial, sqlite might be unable to handle something like monitoring all the ports on those switches.
cpu & ram specs should be more than enough, but make sure your disk subsystem is as fast as possible.
monitor your cache buffers in zabbix (see manual) and keep an eye on the queue - if it raises a lot, you might have to adjust your config (for example, increase polling intervals), or move to another db.
make sure to use latest stable zabbix version, it should be much better performance wise.
note that zabbix appliance has been released today (http://www.zabbix.com/download.php) - it provides operating system, database and zabbix server preconfigured - maybe that's an easier path for you (though that's just the initial appliance release, not suggested for production use yet). you could use hdd image & install it on your server, then see how it works/doesn't :)