Description of problem: After rebalance operation is performed on a volume , logs containg a message from syncop which says " re-running already running task". 2012-04-26 07:26:14.635645] I [client-handshake.c:456:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 [2012-04-26 07:26:14.637349] I [client-handshake.c:1631:select_server_supported_programs] 0-test-client-2: Using Program GlusterFS 3git, Num (1298437), Version (330) [2012-04-26 07:26:14.637769] I [client-handshake.c:1428:client_setvolume_cbk] 0-test-client-2: Connected to 10.16.157.66:24011, attached to remote volume '/home/bricks/b3'. [2012-04-26 07:26:14.637803] I [client-handshake.c:1440:client_setvolume_cbk] 0-test-client-2: Server and Client lk-version numbers are not same, reopening the fds [2012-04-26 07:26:14.637973] I [client-handshake.c:456:client_set_lk_version_cbk] 0-test-client-2: Server lk version = 1 [2012-04-26 07:26:14.643342] I [dht-common.c:2346:dht_setxattr] 0-test-dht: fixing the layout of / [2012-04-26 07:26:14.644318] I [dht-rebalance.c:1067:gf_defrag_migrate_data] 0-test-dht: migrate data called on / [2012-04-26 07:26:14.647059] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.648060] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.651080] I [dht-rebalance.c:648:dht_migrate_file] 0-test-dht: /7: attempting to move from test-client-0 to test-client-2 [2012-04-26 07:26:14.664415] I [dht-rebalance.c:857:dht_migrate_file] 0-test-dht: completed migration of /7 from subvolume test-client-0 to test-client-2 [2012-04-26 07:26:14.665339] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.666262] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.668721] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.669881] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.671025] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.672156] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.673245] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.675153] W [syncop.c:40:__run] 0-glusterfs: re-running already running task [2012-04-26 07:26:14.678629] I [dht-rebalance.c:1623:gf_defrag_status_get] 0-glusterfs: Rebalance is completed [2012-04-26 07:26:14.678655] I [dht-rebalance.c:1626:gf_defrag_status_get] 0-glusterfs: Files migrated: 1, size: 0, lookups: 11, failures: 0 [2012-04-26 07:26:14.679145] W [glusterfsd.c:794:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x350cee570d] (-->/lib64/libpthread.so.0() [0x350d2077f1] (-->/usr/local/sbin/glusterfs(glusterfs_sigwaiter+0xfc) [0x407905]))) 0-: r
Pranith, Can you take a look on this? If re-running the task is an issue, we need to fix, if its not an issue, then logs should be suppressed.
yep there seems to be a problem, env->run_count is going -ve for some reason. I need to debug a bit more. Pranith
CHANGE: http://review.gluster.com/3249 (libgluster: Fix counting, synctask state errors) merged in master by Vijay Bellur (vijay)
No such logs are seen now on 3.3.0qa41