Description of problem: Since some time iSCSI target is not functioning. After some initial I/O, target throws "fd_do_rw() write returned -22" and initiator throws IO error. Detailed configuration steps follow. TARGET (iqn.2003-01.org.linux-iscsi.seurat.x8664, 192.168.22.1) : Rootfs is btrfs (rw,relatime,seclabel,compress=zlib,ssd,space_cache,autodefrag) 1. truncate -s 800M /iscsi.bs0 2. targetcli: cd backstores/fileio create file_or_dev=/iscsi.bs0 name=fbs0 cd /iscsi create cd iqn.2003-01.org.linux-iscsi.seurat.x8664:sn.513490d29dc7/tpg1 \ cd luns create /backstores/fileio/fbs0 cd /iscsi/iqn.2003-01.org.linux-iscsi.seurat.x8664:sn.513490d29dc7/tpg1/acls create iqn.2003-01.org.linux-iscsi.ginjo.x8664:sn.37648a3d9cbe ls / o- / ............................................................................ [...] o- backstores ................................................................. [...] | o- block ..................................................... [Storage Objects: 0] | o- fileio .................................................... [Storage Objects: 1] | | o- fbs0 ............................ [/iscsi.bs0 (800.0MiB) write-back activated] | o- pscsi ..................................................... [Storage Objects: 0] | o- ramdisk ................................................... [Storage Objects: 0] | o- user ...................................................... [Storage Objects: 0] o- iscsi ............................................................... [Targets: 1] | o- iqn.2003-01.org.linux-iscsi.seurat.x8664:sn.513490d29dc7 ............. [TPGs: 1] | o- tpg1 .................................................. [no-gen-acls, no-auth] | o- acls ............................................................. [ACLs: 1] | | o- iqn.2003-01.org.linux-iscsi.ginjo.x8664:sn.37648a3d9cbe . [Mapped LUNs: 1] | | o- mapped_lun0 .................................... [lun0 fileio/fbs0 (rw)] | o- luns ............................................................. [LUNs: 1] | | o- lun0 .......................................... [fileio/fbs0 (/iscsi.bs0)] | o- portals ....................................................... [Portals: 1] | o- 0.0.0.0:3260 ........................................................ [OK] INITIATOR (iqn.2003-01.org.linux-iscsi.ginjo.x8664:sn.37648a3d9cbe, 192.168.22.3) : 1. iscsiadm -m discovery -t st -p 192.168.22.1 192.168.22.1:3260,1 iqn.2003-01.org.linux-iscsi.seurat.x8664:sn.513490d29dc7 2. iscsiadm -m node -l [May 6 08:05] scsi host4: iSCSI Initiator over TCP/IP [ +0.041362] scsi 4:0:0:0: Direct-Access LIO-ORG fbs0 4.0 PQ: 0 ANSI: 5 [ +0.028042] sd 4:0:0:0: Attached scsi generic sg2 type 0 [ +0.014333] sd 4:0:0:0: [sdb] 1638400 512-byte logical blocks: (838 MB/800 MiB) [ +0.010611] sd 4:0:0:0: [sdb] Write Protect is off [ +0.003731] sd 4:0:0:0: [sdb] Mode Sense: 43 00 10 08 [ +0.013365] sd 4:0:0:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA [ +0.083011] sd 4:0:0:0: [sdb] Attached SCSI disk 3. mkfs.btrfs -M -L seurat-iscsi /dev/sdb 4. btrfstune -x /dev/sdb 5. mount /dev/sdb /mnt/tmp -o autodefrag,compress=lzo [May 6 08:06] BTRFS: device label seurat-iscsi devid 1 transid 3 /dev/sdb [ +22.736061] BTRFS info (device sdb): disk space caching is enabled [ +0.003400] BTRFS: has skinny extents [ +0.010614] BTRFS: creating UUID tree [May 6 08:07] BTRFS info (device sdb): enabling auto defrag [ +0.005219] BTRFS info (device sdb): setting 8 feature flag [ +0.003252] BTRFS info (device sdb): disk space caching is enabled (there are two mounts in the log snippet above) NOW START THE I/O. Soon: INITIATOR: [May 6 08:08] fd_do_rw() write returned -22 [ +0.107407] fd_do_rw() write returned -22 TARGET: [May 6 08:08] sd 4:0:0:0: [sdb] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ +0.004159] sd 4:0:0:0: [sdb] Sense Key : Not Ready [current] [ +0.001032] sd 4:0:0:0: [sdb] Add. Sense: Logical unit communication failure [ +0.003389] sd 4:0:0:0: [sdb] CDB: Write(10) 2a 00 00 01 6b 20 00 2b e0 00 [ +0.003223] blk_update_request: I/O error, dev sdb, sector 92960 [ +0.001409] BTRFS: bdev /dev/sdb errs: wr 1, rd 0, flush 0, corrupt 0, gen 0 [ +0.002002] BTRFS: bdev /dev/sdb errs: wr 2, rd 0, flush 0, corrupt 0, gen 0 [ +0.000759] BTRFS: bdev /dev/sdb errs: wr 3, rd 0, flush 0, corrupt 0, gen 0 [ +0.001236] BTRFS: bdev /dev/sdb errs: wr 4, rd 0, flush 0, corrupt 0, gen 0 [ +0.000801] BTRFS: bdev /dev/sdb errs: wr 5, rd 0, flush 0, corrupt 0, gen 0 [ +0.000644] BTRFS: bdev /dev/sdb errs: wr 6, rd 0, flush 0, corrupt 0, gen 0 [ +0.088333] sd 4:0:0:0: [sdb] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ +0.000003] sd 4:0:0:0: [sdb] Sense Key : Not Ready [current] [ +0.000003] sd 4:0:0:0: [sdb] Add. Sense: Logical unit communication failure [ +0.000003] sd 4:0:0:0: [sdb] CDB: Write(10) 2a 00 00 00 a0 00 00 3b 20 00 [ +0.000003] blk_update_request: I/O error, dev sdb, sector 40960 [ +0.000006] BTRFS: bdev /dev/sdb errs: wr 7, rd 0, flush 0, corrupt 0, gen 0 [ +0.000195] BTRFS: bdev /dev/sdb errs: wr 8, rd 0, flush 0, corrupt 0, gen 0 [ +0.000085] BTRFS: bdev /dev/sdb errs: wr 9, rd 0, flush 0, corrupt 0, gen 0 [ +0.000082] BTRFS: bdev /dev/sdb errs: wr 10, rd 0, flush 0, corrupt 0, gen 0 Application exits with I/O errors, btrfs sometimes switches to read-only. Version-Release number of selected component (if applicable): TARGET: kernel-core-4.0.1-300.fc22.x86_64 targetcli-2.1.fb40-1.fc21.noarch INITIATOR: kernel-core-4.1.0-0.rc1.git1.1.fc23.x86_64 iscsi-initiator-utils-6.2.0.873-25.gitc9d830b.fc22.x86_64 How reproducible: I've reproduced it twice on different hosts/VMs. In both cases initiator/target are directly connected: a) target on physical host, initiator is a VM hosted on this host - KVM is used; b) target and initiator are VMs hosted on the same machine, VirtualBox is used.
Created attachment 1022488 [details] saveconfig.json
I think I met the same problem. So,e digging around and I decided to give a try to this patch: https://git.kernel.org/cgit/linux/kernel/git/gregkh/staging.git/commit/?h=staging-testing&id=6b9a44d0939ca6235c72b811bd55b462d6a0a553 With this in-place my problems seems to be all gone now, or so it seems. If you wnat to give it a try, but don't want to build the RPM yourself, here is the one I built: http://crimea.edu/~green/kernel/
I just want to add that the patch is to the target, not to initiator. so the patched rpm is that of the patched current F22 kernel.
kernel-4.0.5-300.fc22 has been submitted as an update for Fedora 22. https://admin.fedoraproject.org/updates/kernel-4.0.5-300.fc22
kernel-4.0.5-200.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/kernel-4.0.5-200.fc21
Package kernel-4.0.5-200.fc21: * should fix your issue, * was pushed to the Fedora 21 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing kernel-4.0.5-200.fc21' as soon as you are able to, then reboot. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2015-9704/kernel-4.0.5-200.fc21 then log in and leave karma (feedback).
kernel-4.0.5-200.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report.
kernel-4.0.5-300.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.