How do you remove the pump but leave the shaft? You need to take pictures of what's going on under that cover. If you want to be rid of the oil injection system, you either need to remove the gear from the shaft or buy an injection delete kit.
First thing, the oil pump isn't only mounted to the clutch cover which is holding in the transmission fluid. Even if the shaft is leaking, it's not allowing an "air leak" into the motor, only allowing sloshing transmission fluid to seep out.