Bug 1614654

Description of problem:

From the runs captured at https://review.gluster.org/#/c/20637/, I saw that the latest runs where this particular .t failed were at https://build.gluster.org/job/line-coverage/415 and https://build.gluster.org/job/line-coverage/421/.
In both of these runs, there are no gluster 'regression' logs available at https://build.gluster.org/job/line-coverage/<build-number>/artifact. I have raised BZ 1613721 for it.

Jenkins output is:
20:37:43 ./tests/basic/afr/add-brick-self-heal.t .. 
20:37:43 1..34
20:37:43 ok 1, LINENUM:6
20:37:43 ok 2, LINENUM:7
20:37:43 ok 3, LINENUM:8
20:37:43 ok 4, LINENUM:9
20:37:43 ok 5, LINENUM:10
20:37:43 ok 6, LINENUM:11
20:37:43 ok 7, LINENUM:12
20:37:43 ok 8, LINENUM:14
20:37:43 ok 9, LINENUM:15
20:37:43 ok 10, LINENUM:24
20:37:43 ok 11, LINENUM:27
20:37:43 ok 12, LINENUM:30
20:37:43 ok 13, LINENUM:31
20:37:43 ok 14, LINENUM:34
20:37:43 ok 15, LINENUM:35
20:37:43 ok 16, LINENUM:36
20:37:43 ok 17, LINENUM:38
20:37:43 ok 18, LINENUM:39
20:37:43 ok 19, LINENUM:40
20:37:43 ok 20, LINENUM:42
20:37:43 ok 21, LINENUM:43
20:37:43 ok 22, LINENUM:44
20:37:43 ok 23, LINENUM:45
20:37:43 ok 24, LINENUM:46
20:37:43 ok 25, LINENUM:47
20:37:43 not ok 26 Got "10" instead of "^0$", LINENUM:50
20:37:43 FAILED COMMAND: ^0$ get_pending_heal_count patchy
20:37:43 ok 27, LINENUM:53
20:37:43 ok 28, LINENUM:54
20:37:43 not ok 29 , LINENUM:57
20:37:43 FAILED COMMAND: diff /d/backends/patchy0/file1.txt /d/backends/patchy2/file1.txt
20:37:43 not ok 30 Got "" instead of "qwerty", LINENUM:60
20:37:43 FAILED COMMAND: qwerty get_text_xattr user.test /d/backends/patchy2/file5.txt
20:37:43 ok 31, LINENUM:61
20:37:43 ok 32, LINENUM:63
20:37:43 ok 33, LINENUM:64
20:37:43 ok 34, LINENUM:65
20:37:43 Failed 3/34 subtests 
20:37:43 


So self-heal has failed to heal the entries. This could be due to bricks not being up/ shd not able to connect to the bricks/ lock contention and wait for $HEAL_TIMEOUT as described in BZ 1613807.

Hence modifying the .t to check for brick up status and reducing heal timeout to 5 seconds.


Note:
After the above said runs, the test has not failed again, up to the latest revision 8 of https://review.gluster.org/#/c/2063. So sending the patch with whatever can be theoretically fixed in the .t