Per Amit,I clone the bug to rhel6.5 in case of any potential issues.

Tried the bug on build qemu-kvm-0.12.1.2-2.382.el6.x86_64 but I didn't reproduce the issue.
Steps,
  1./usr/libexec/qemu-kvm -m 4096 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -name usb-device -uuid b03eea94-a502-4142-b541-96f86473a07a -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0 -chardev pty,id=channel1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm1,bus=virtio-serial0.0,id=port1,nr=1 -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm2,bus=virtio-serial0.0,id=port2,nr=2 -drive  file=/home/rhel64-new.raw,if=none,id=drive-system-disk,format=raw,cache=none,aio=native,werror=stop,rerror=stop,serial=QEMU-DISK1 -device ide-drive,bus=ide.0,unit=0,drive=drive-system-disk,id=system-disk,bootindex=1 -netdev tap,sndbuf=0,id=hostnet0,script=/etc/qemu-ifup,downscript=no -device e1000,netdev=hostnet0,mac=00:15:65:01:3a:20 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -vnc :1 -monitor stdio -qmp tcp:0:4444,server,nowait
  2.remove port1
  3.{"execute": "qmp_capabilities"}
{"return": {}}
{"execute": "chardev-remove", "arguments": { "id" : "channel1" } }
{"return": {}}
 
  Actual results,the guest worked well  without segmentation fault
  Expected results,the guest works well without segmentation fault

Doesn't reproduce on rhel6, lowering priority.
Waiting for rhel7 bug #994414 being analyzed.

Can not reproduce this issue on qemu-kvm-0.12.1.2-2.409.el6.x86_64.

host:
RHEL6.5-Snapshot-2.0
qemu-kvm-tools-0.12.1.2-2.409.el6.x86_64
gpxe-roms-qemu-0.9.7-6.10.el6.noarch
qemu-kvm-debuginfo-0.12.1.2-2.409.el6.x86_64
qemu-img-0.12.1.2-2.409.el6.x86_64
qemu-kvm-0.12.1.2-2.409.el6.x86_64

guest:
kernel-2.6.32-358.el6.x86_64

steps
1.boot up guest with CLI
/usr/libexec/qemu-kvm \
-M pc \
-cpu Nehalem \
-m 4096 \
-smp 4,sockets=2,cores=2,threads=1,maxcpus=16 \
-enable-kvm \
-name rhel6u5 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-rtc base=localtime,clock=host,driftfix=slew \
-no-kvm-pit-reinjection \
-monitor stdio \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-qmp tcp:0:6666,server,nowait \
-nodefconfig \
-nodefaults \
-global PIIX4_PM.disable_s3=0 \
-global PIIX4_PM.disable_s4=0 \
-vga qxl \
-spice port=5900,disable-ticketing \
-drive file=/home/rhel6u5.qcow2,if=none,id=gfs0,cache=none,aio=native \
-device virtio-blk-pci,drive=gfs0,bootindex=0 \
-device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0 \
-chardev pty,id=channel1,server,nowait \
-device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm1,bus=virtio-serial0.0,id=port1,nr=1 \
-chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait \
-device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm2,bus=virtio-serial0.0,id=port2,nr=2 \

2. remove port1
(qemu) device_del port1

3.remove chardev channel1
[root@m2 ~]# telnet localhost 6666
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
{"QMP": {"version": {"qemu": {"micro": 1, "minor": 12, "major": 0}, "package": "(qemu-kvm-0.12.1.2)"}, "capabilities": []}}
{"execute": "qmp_capabilities"}
{"return": {}}
{"execute": "chardev-remove", "arguments": { "id" : "channel1" } }
{"return": {}}

Result:
Guest works well, no crash or segmentation fault.

Update qemu-kvm to qemu-kvm-0.12.1.2-2.412.el6.x86_64, also can not hit this issue.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-1553.html