Often someone “on the outside looking in” can have a better approach or solution to a problem in your relationship.
Not only can you lose yourself if you spend all your time with a guy, you also lose the time you'd spend meeting other guys and potential lifelong friends.
College sex is almost always awkward sex -- because getting it on in an elevator or stairwell, while back then just part of fall semester, is something I think we can (mostly) agree is pretty awful. You're out on your own for the first time, traversing the sea of collegiate genitals and unfettered freedom.
We look back on those weird experiences with fondness; and an ever-so-slight feeling of discomfort. Sometimes you just get so bored of the same old roid-raging douche kabobs at school and just need to explore other terrain.
If you've been drinking, there's not always a trusted friend there to stop you from going too far with a guy you just met.
Just like there might be things you're not telling him about your life, he could be hiding stuff too.