Memory management is already a complex topic, and unfortunately, Microsoft has altered its exact definitions and terms regarding different types of memory with pretty much every single release of Windows. So trust me, you aren't the only one confused by it.
When you use tasklist, tasklist shows you the Working Set that is assigned to each process. A working set is the recently-touched physical pages of RAM assigned to a process, and that does not account for all the total virtual memory that a process may be using (in a sense, processes only use virtual memory, as physical memory is abstracted into a larger pool of virtual memory and combined with other sources of virtual memory such as the page file.) Nor does it account for file caching, etc.
You should never have much "Free" memory on your system. That is wasted RAM that is literally doing nothing. When there are no processes that need any more memory, Windows uses the free memory to cache files, until the Free memory is back down to zero or close to zero.
But that cache can be very quickly reallocated to a process, at which point it would become part of that processes Virtual Bytes and subtracted from available memory.
As for your slowdowns, you could be experiencing a lot of things. When Windows gets low on available memory, it starts eating away (or trimming) processes working sets, which can lead to noticeable slowdowns.
Use the descriptions found in Performance Monitor to get the most accurate descriptions when you're talking about these types of memory. (As opposed to Task Manager, which has some conflicting/confusing terminology for some of the terms.)
Lastly, I highly recommend that you watch this two-part video series: http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/WCL405