That's it, really. If you change the IP address with linuxconf, it doesn't restart nmbd, so the machine still answers nmb queries with its old IP address.
Added Jacques to CC line I'm not sure how to correctly resolve this. Should changing an IP address automatically restart most network services? There is not a well-defined dependency graph for this...
There's no good way to do this.