Fix potential core dump when closing ports at exit