How do I install gcc inside of a VPS?

You must install the -devel package for that OS template. For
example, for VPS #101 based on Fedora Core 2, run the following command
on the hardware node:

# vzpkgadd 101 devel-fc2