Description of problem: Seeing I/O error eventaul I/O timeout while failing hot tier Version-Release number of selected component (if applicable): glusterfs-server-3.7dev-0.994.git0d36d4f.el6.x86_64 Volume Name: vol1 Type: Tier Volume ID: b77d4050-7fdc-45ff-a084-f85eec2470fc Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: 10.70.35.67:/rhs/brick2 Brick2: 10.70.35.56:/rhs/brick2 Brick3: 10.70.35.56:/rhs/brick1 Brick4: 10.70.35.67:/rhs/brick1 Steps to Reproduce: 1.Create a tiered volume 2. Start I/O, 100% writes to the volume 3. While the I/O is in progress, fail the bricks in hot tier Actual results: bzip2: I/O or other error, bailing out. Possible reason follows. bzip2: Transport endpoint is not connected Input file = (stdin), output file = (stdout) tar: linux-2.6.31.1/arch/ia64/kernel/gate.lds.S: Cannot open: No such file or directory linux-2.6.31.1/arch/ia64/kernel/head.S tar: linux-2.6.31.1/arch/ia64/kernel/head.S: Cannot open: No such file or directory linux-2.6.31.1/arch/ia64/kernel/ia64_ksyms.c tar: linux-2.6.31.1/arch/ia64/kernel/ia64_ksyms.c: Cannot open: No such file or directory linux-2.6.31.1/arch/ia64/kernel/init_task.c tar: linux-2.6.31.1/arch/ia64/kernel/init_task.c: Cannot open: No such file or directory linux-2.6.31.1/arch/ia64/kernel/iosapic.c tar: linux-2.6.31.1/arch/ia64/kernel/iosapic.c: Cannot open: No such file or directory tar: Child returned status 1 Expected results: Even if the hot tier is not present, the fresh writes should be routed to the cold tier. *Looks like all new write are default routed to the hot tier.
This problem is basically the same as 1212822. We do not keep copies of files on both tiers and it is expected behavior for the file not to be found if all the bricks in the hot tier are taken down. This is why you would want redundancy on the hot tier. We can discuss further to iron out any details in the understanding, so I'll put this ON_QA for now.
Reproduced this ont the BETA2 build too, hence moving it to ASSIGNED.
Even if it is a fresh write we cannot create a new file. We cannot know if the file already exists on the hot tier. If it turned out the file did already exist, and we created a second file with the same name on the cold, this would overwrite the old file and be dataloss. This is not a bug.