Sort of damned if you do, and damned if you don't. (I know, a trite response).
There is no cure for the common cold.. (another bit of genius advice, right?).. So what are the actual benefits of going? Well, first off, as you said, you are on immuno suppressors... it could turn serious PDQ. how good is your doctor? How long does it take to get an appointment? What are you doing... taking your temperature, scaling/noting your symptoms, a 'packed' bag with enuff essentials/info in case it means a trip to the ER, and possibly an overnite or longer stay in the hospital? sort of like planning for a 'natural' disaster, except on a single person scale. OTC products do so little... alleviate symptoms that could be telling you something 'vital'. If possible, limit your use of them... why add to the chemo mix in your body IF you can avoid it. Get plenty of rest, take it easy, eat totally right for your situation (not the best time to play fast and loose, or is that lose? Or is it play fast N loose and 'lose'?). Then, allow time for things to run natural course... 'They' say a cold runs 3 - 7 days.. I dunno. Maybe its just my advanced years, but most colds stay with me 7 - 10 days. Flu takes a bit longer... usually symptoms move downward.. chest congestion, stomach issues, more intense pains (but those could be std IBD symptoms). Thing to watch for is that you don't just lie down and accept it. Any chest inflamation can turn into pneumonia IF you allow congestion to build up too much... Really... If you enter your 3rd week, day 15 plus, and aren't feeling better, or if fever or other symptoms you can AND do measure take a drastic turn for the worse, get to see a professional, your GP or emerg dr ASAP. That's all pretty self evident advice, no rocket science there at all, but.. wer'e living (or trying to) with a lifelong chronic illness, we all either have compromised immune systems, or else take drugs that do it for us, and we have to play it safe.. Problem is, if you dont' have something really life threatening going into a dr's office or emergency room, chances are (and it's no exxageration, I've been exposed to a/b resistant strains of bugs on 75% of my hospital stays - even got red flagged for a while - no joke) you WILL be exposed to something really nasty in one place or the other..
so, that's the end of my long winded and almost completely useless reply to your question. One point I would strongly disagree with is that theres' no such thing as a 'silly' question. Crohns'/IBD is too serious an issue for any of our questions to be considered 'silly'.. And you know who would probably give you the 'best' answer to this REAL, LEGITIMATE question? your doctor.. SERIOUSLY! Why not pose this to him/her at your next visit. That way you'll get an informed, professional answer to it from someone who is intimately familiar with your medical background and crohns situation.