The toga was unique to Rome. It was wrapped and draped around the body over a tunic. Wearing a toga was a privilege reserved for adult male Roman citizens. When the son of a citizen reached puberty, he received his first toga as a rite of passage. Stripes of color on the border of a toga were emblems of political rank.
But not all eligible citizens had togas. A toga was expensive to start with, cumbersome to wear, and almost impossible to don without help. So in actual practice only wealthy citizens who could afford slaves wore togas. Everyone else just wore the tunic. Married women wore a garment called a over their tunics. It was wrapped and draped almost like a toga, but was lighter and less cumbersome.
If you're planning a college toga party you may not care about these historical niceties. That's not what a toga party is about! A toga party is about re-creating the decadence and perhaps debauchery of the waning Roman Empire. So have as much fun as you want. Just don't try to pass it off as historically accurate.