Clone for GlusterFS 3.6 +++ This bug was initially created as a clone of Bug #1136221 +++ Description of problem: We construct some rpc messages and send it to the IP and port which glusterfsd listens, the memory usage goes up quickly until exhausted Version-Release number of selected component (if applicable): 3.3.0, 3.4.1, 3.5.0 Steps to Reproduce: 1. Start glusterfs services, and get the IP and port that one glusterfsd process listens 2. Run the attachement python script, which connects the IP and port and send four bytes 00 00 00 00 to the glusterfsd process 3. Watch the memory usage of the glusterfsd process. It will grow up quickly Actual results: Memory of the glusterfsd process grows up quickly till exhausted Expected results: Glusterfsd just ignores the messages Additional info: The bug seems in __socket_proto_state_machine, which goes into an infinite loop to malloc memories when handle the special message. The special message is "multi fragments in a single record", and some values are not reset when handle next fragment. We tested below fix and it seems work: if (!RPC_LASTFRAG (in->fraghdr)) { + in->pending_vector = in->vector; + in->pending_vector->iov_base = &in->fraghdr; + in->pending_vector->iov_len = sizeof (in->fraghdr); in->record_state = SP_STATE_READING_FRAGHDR; break; } --- Additional comment from jiangkai on 2014-09-04 06:35:44 EDT --- More issues than imaging to handle the "multi fragments in a single record" message. The proposal is to refuse it: if (!RPC_LASTFRAG (in->fraghdr)) { gf_log (this->name, GF_LOG_ERROR, "multiple fragments per record not supported now"); ret = -1; goto out; } --- Additional comment from jiangkai on 2014-09-05 04:45:08 EDT --- It happens after 3.4; 3.3.1 reports error messages. It seems imported by the change id Icd9f256bb2fd8c6266a7abefdff16936b4f8922d to support SSL --- Additional comment from Anand Avati on 2014-09-09 04:30:12 EDT --- REVIEW: http://review.gluster.org/8662 (socket: Fixed parsing RPC records containing multi fragments) posted (#1) for review on master by Gu Feng (flygoast) --- Additional comment from Niels de Vos on 2014-09-11 05:54:58 EDT --- http://review.gluster.org/8662 is for mainline bug 1139598, we can backport the change when it has been merged in the master branch.
REVIEW: http://review.gluster.org/8838 (socket: Fixed parsing RPC records containing multi fragments) posted (#1) for review on release-3.6 by Vijay Bellur (vbellur)
COMMIT: http://review.gluster.org/8838 committed in release-3.6 by Raghavendra G (rgowdapp) ------ commit b7d2482d85d57b4eaf2110060f62f17769895da7 Author: Gu Feng <flygoast> Date: Tue Sep 9 18:00:22 2014 +0800 socket: Fixed parsing RPC records containing multi fragments In __socket_proto_state_machine(), when parsing RPC records containing multi fragments, just change the state of parsing process, had not processed the memory to coalesce the multi fragments. Change-Id: I5583e578603bd7290814a5d26885b31759c73115 BUG: 1146200 Signed-off-by: Gu Feng <flygoast> Reviewed-on: http://review.gluster.org/8662 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Niels de Vos <ndevos> Reviewed-by: Raghavendra G <rgowdapp> Tested-by: Raghavendra G <rgowdapp> Reviewed-on: http://review.gluster.org/8838
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report. glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html [2] http://supercolony.gluster.org/mailman/listinfo/gluster-users