I find the simplest way to do this is to set up an ssh proxy command, which lets me ssh directly to server B from my workstation, bouncing through server A.
In my .ssh/config file on my workstation, I add the following:
ProxyCommand ssh serverA nc -q 0 IP_of_serverB 22
What this says is that when you connect to a host called "serverB", it will actually set up an ssh connection to serverA first, where it will run "nc" (netcat) which will dump all traffic into port 22 on IP_of_serverB.
When you connect, you first of all make an ssh proxy connection to serverA (so it will ask for your serverA password), and then it makes the real ssh connection to serverB (so it will ask for your serverB password).
Once this works, you can scp directly to serverB, and it will proxy through serverA
This requires that you have the tool "netcat" (/bin/nc) installed on serverA. If you don't have that, there are probably other tools you can use instead, try doing a google search for ProxyCommand
There are other ways you can build up that config, with ssh_config macros and so on, however I find the above format to be the most reliable.
If you have different usernames on serverA and serverB, you can set the usernames in the config file as well. Check the ssh_config manpage, and google for "ssh ProxyCommand" for more information.