It would help if your question was a little bit more specific, but I'll give it a go.
Some will point out that this is not sysadmin related, but I disagree. Setting up development environments for programmers and providing support is a part of the job for many of us.
Pick a distribution and install it. You could try a live-cd, but not many development environments can be run from that.
Know what you're going to program for, is it strictly a Linux program? Is it a desktop program?
Select your toolkit and libraries.
Check out Stack Overflow, the programming related sister-site of this web page.
Starting on Linux (from my experience) can be difficult. But remember, while Linux may not have the biggest user share, almost every single user, posts his problems on an open forum. This means, you can Google (search) for pretty much any problem you can think of. In my opinion, it is easier to find help online for Linux, than it is for Windows (depending on your distribution, Ubuntu recommended for beginners).
Pick an IDE (Development Environment), I for example like Geany (that comes with Gnome/Ubuntu) but numerous alternatives exist.
It's not a very specific answer, but the question wasn't very specific so I hope it will point you in the right direction. Most of us "Linux dudes" were Windows users at some point and I believe that your logical thinking and math degree will help you out.
Just think 'Matlab' if you're ever stuck in the terminal.