I was originally diagnosed with Crohn's in 1999 because of a fistula and abscess close to the rectum. It took over a year to finally subside and of course it's still there 14 years later (but no problem any more).
As with yours, it "leaked" through on a regular basis back then. My doc back then and the surgeon said an operation to remove the abscess would not be good. Instead they drained and washed the fistula (quite painful because for some reason they didn't put me under anestetics...) and within a few weeks from that it sort of dried up and was fine.
But to be honest, I am no expert in fistula management as this was quite some time ago and I didn't have any new fistulas developing since then. I also got meds (antibiotics) that supposedly help with fistulas, but no clue if they really helped.