It's worth the effort if you have the will to put in the effort. If the time is not right for you, don't go. I don't think it's ever "too late" to go to college. You should go when your mind is in the mindset.
When I went to university, I wish I had been in a better mindset during my five years there. At times, I feel as if I walked away with very little. I know I had a lot of life experiences as a result of going, but education-wise, because I went into a major that my heart wasn't feeling, I graduated feeling like I got very little out of the money and time I put into it. If I had a second chance, I would have applied for the major I knew I would have loved from the very beginning. I went into something because my parents said it'd be good for me, and now I'm graduated with a degree I don't really plan to use.
If you do go to college, be sure you are going in for what you love, because your experience while learning will be a lot better. Or even if you do happen to change your major some part of the way in, that's ok, too.
Even if you go in undeclared, it's ok. I usually think it is ok to not know what you love, but it's also useful to know what you don't like.
As long as you're a high school graduate, I think it's ok to go do other stuff and explore and get some REAL LIFE experience before going to university.
It's worth all that time, effort and knowledge in the long run if going to college is something you really want to do and put your 100% into your education, like studying and making connections with other people. You definitely can't go it alone. I think that is also another reason why my experience sucked. I was pretty depressed for most of it and so I had zero friends and graduated with zero.
I hope you make the choice that's right for you, whatever's in your heart!