Buy a small wooden dowel at Lowe’s/Home Depot. One that will fit in the hole with a bit of room to spare. Cut about 12″, then use a band saw to slice a slot in one end (make it look like an old fashioned tinker-toy stick if you know what that is.) Chuck this up in a variable speed drill, then take a good piece of emory cloth, cut a strip about 1-2″ wide, about the same length, and slide it into this slot. Wrap it around the dowel so that the rough side is out, insert this into the bushing and use the drill to spin it. This will open the thing up, while maintaining a smooth finish. Doesn’t take long. You can smooth it further by going to 400 grit or finer, but if you start with 400, expect to be a while.

I’d likely start with 240.
Doing it like this, you can get the fit to be “just right” if you want it to have zero slop but not be impossible to turn the post to tighten/loosen it.