Bug 1643326 - Storage domain mount error: Lustre file system (Posix compliant FS)
Summary: Storage domain mount error: Lustre file system (Posix compliant FS)
Keywords:
Status: CLOSED DUPLICATE of bug 1592916
Alias: None
Product: vdsm
Classification: oVirt
Component: General
Version: 4.20.31
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
: ---
Assignee: Dan Kenigsberg
QA Contact: Elad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-10-26 01:59 UTC by okok102928
Modified: 2018-11-05 09:36 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-05 09:36:36 UTC
oVirt Team: Storage
Embargoed:


Attachments (Terms of Use)

Description okok102928 2018-10-26 01:59:27 UTC
Description of problem:
As subject


Version-Release number of selected component (if applicable):
vdsm : 4.20.31-1.el7ev
ovirt-engine : 4.2.4.5-0.1.el7_3
libvirt : 3.9.0-14.el7_5.6
sanlock : 3.6.0-1.el7


How reproducible:
100%


Steps to Reproduce:
1. Install the hypervisor and RHVM with the image rhvh-4.2.4.3-0.20180622 (I think the version does not matter)

2. Add storage domain configured with luster file system with Posix-compatible FS
(Uid, gid must be mapped to the storage server)

Domain Function: Data
Storage Type: POSIX compliant FS
Host to Use: [SPM_HOSTNAME]
Name: [STORAGE_DOMAIN_NAME]
Path: [STORAGE_MANAGEMENT_IP]@tcp:[FSNAME(like volume name)]
VFS Type: luster
Mount Options:

3. rhvm WEB pops up the message.

Error while executing action AddPosixFsStorageDomain: Error creating a storage domain

engine.log : 
2018-10-26 10:50:36,956+09 INFO  [org.ovirt.engine.core.bll.storage.connection.AddStorageServerConnectionCommand] (default task-61) [5f2e2f52-eab8-4c16-a5cd-727d492f855f] Lock Acquired to object 'EngineLock:{exclusiveLocks='[10.10.10.15@tcp:/lustre/vmstore=STORAGE_CONNECTION]', sharedLocks=''}'
2018-10-26 10:50:36,963+09 INFO  [org.ovirt.engine.core.bll.storage.connection.AddStorageServerConnectionCommand] (default task-61) [5f2e2f52-eab8-4c16-a5cd-727d492f855f] Running command: AddStorageServerConnectionCommand internal: false. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN
2018-10-26 10:50:36,965+09 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (default task-61) [5f2e2f52-eab8-4c16-a5cd-727d492f855f] START, ConnectStorageServerVDSCommand(HostName = host3.intel.dom, StorageServerConnectionManagementVDSParameters:{hostId='ba7e5336-2884-498e-8845-44bc17f5ab3b', storagePoolId='00000000-0000-0000-0000-000000000000', storageType='POSIXFS', connectionList='[StorageServerConnections:{id='null', connection='10.10.10.15@tcp:/lustre/vmstore', iqn='null', vfsType='lustre', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 24d0899f
2018-10-26 10:50:37,056+09 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (default task-61) [5f2e2f52-eab8-4c16-a5cd-727d492f855f] FINISH, ConnectStorageServerVDSCommand, return: {00000000-0000-0000-0000-000000000000=0}, log id: 24d0899f
2018-10-26 10:50:37,063+09 INFO  [org.ovirt.engine.core.bll.storage.connection.AddStorageServerConnectionCommand] (default task-61) [5f2e2f52-eab8-4c16-a5cd-727d492f855f] Lock freed to object 'EngineLock:{exclusiveLocks='[10.10.10.15@tcp:/lustre/vmstore=STORAGE_CONNECTION]', sharedLocks=''}'
2018-10-26 10:50:37,094+09 INFO  [org.ovirt.engine.core.bll.storage.domain.AddPosixFsStorageDomainCommand] (default task-20) [47702dc7-1973-45a4-8116-a4e7d8b80386] Running command: AddPosixFsStorageDomainCommand internal: false. Entities affected :  ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN
2018-10-26 10:50:37,127+09 INFO  [org.ovirt.engine.core.bll.profiles.AddDiskProfileCommand] (default task-20) [18e068c6] Running command: AddDiskProfileCommand internal: true. Entities affected :  ID: 28ce0e8d-1073-4e82-99a4-0be72a508696 Type: StorageAction group CREATE_STORAGE_DISK_PROFILE with role type ADMIN
2018-10-26 10:50:37,137+09 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-20) [18e068c6] EVENT_ID: USER_ADDED_DISK_PROFILE(10,120), Disk Profile vmstore was successfully added (User: admin@internal-authz).
2018-10-26 10:50:37,142+09 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (default task-20) [18e068c6] START, ConnectStorageServerVDSCommand(HostName = host3.intel.dom, StorageServerConnectionManagementVDSParameters:{hostId='ba7e5336-2884-498e-8845-44bc17f5ab3b', storagePoolId='00000000-0000-0000-0000-000000000000', storageType='POSIXFS', connectionList='[StorageServerConnections:{id='f8d72480-9c0e-43ce-9e3b-eeb253dbc5cf', connection='10.10.10.15@tcp:/lustre/vmstore', iqn='null', vfsType='lustre', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 788a5af3
2018-10-26 10:50:37,147+09 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (default task-20) [18e068c6] FINISH, ConnectStorageServerVDSCommand, return: {f8d72480-9c0e-43ce-9e3b-eeb253dbc5cf=0}, log id: 788a5af3
2018-10-26 10:50:37,149+09 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-20) [18e068c6] START, CreateStorageDomainVDSCommand(HostName = host3.intel.dom, CreateStorageDomainVDSCommandParameters:{hostId='ba7e5336-2884-498e-8845-44bc17f5ab3b', storageDomain='StorageDomainStatic:{name='vmstore', id='28ce0e8d-1073-4e82-99a4-0be72a508696'}', args='10.10.10.15@tcp:/lustre/vmstore'}), log id: 7da74c3
2018-10-26 10:50:37,535+09 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-20) [18e068c6] Failed in 'CreateStorageDomainVDS' method
2018-10-26 10:50:37,540+09 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-20) [18e068c6] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM host3.intel.dom command CreateStorageDomainVDS failed: Error creating a storage domain: (u'storageType=6, sdUUID=28ce0e8d-1073-4e82-99a4-0be72a508696, domainName=vmstore, domClass=1, typeSpecificArg=10.10.10.15@tcp:/lustre/vmstore domVersion=4',)
2018-10-26 10:50:37,541+09 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-20) [18e068c6] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand' return value 'StatusOnlyReturn [status=Status [code=351, message=Error creating a storage domain: (u'storageType=6, sdUUID=28ce0e8d-1073-4e82-99a4-0be72a508696, domainName=vmstore, domClass=1, typeSpecificArg=10.10.10.15@tcp:/lustre/vmstore domVersion=4',)]]'
2018-10-26 10:50:37,541+09 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-20) [18e068c6] HostName = host3.intel.dom
2018-10-26 10:50:37,541+09 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-20) [18e068c6] Command 'CreateStorageDomainVDSCommand(HostName = host3.intel.dom, CreateStorageDomainVDSCommandParameters:{hostId='ba7e5336-2884-498e-8845-44bc17f5ab3b', storageDomain='StorageDomainStatic:{name='vmstore', id='28ce0e8d-1073-4e82-99a4-0be72a508696'}', args='10.10.10.15@tcp:/lustre/vmstore'})' execution failed: VDSGenericException: VDSErrorException: Failed to CreateStorageDomainVDS, error = Error creating a storage domain: (u'storageType=6, sdUUID=28ce0e8d-1073-4e82-99a4-0be72a508696, domainName=vmstore, domClass=1, typeSpecificArg=10.10.10.15@tcp:/lustre/vmstore domVersion=4',), code = 351
2018-10-26 10:50:37,541+09 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (default task-20) [18e068c6] FINISH, CreateStorageDomainVDSCommand, log id: 7da74c3
2018-10-26 10:50:37,541+09 ERROR [org.ovirt.engine.core.bll.storage.domain.AddPosixFsStorageDomainCommand] (default task-20) [18e068c6] Command 'org.ovirt.engine.core.bll.storage.domain.AddPosixFsStorageDomainCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to CreateStorageDomainVDS, error = Error creating a storage domain: (u'storageType=6, sdUUID=28ce0e8d-1073-4e82-99a4-0be72a508696, domainName=vmstore, domClass=1, typeSpecificArg=10.10.10.15@tcp:/lustre/vmstore domVersion=4',), code = 351 (Failed with error StorageDomainCreationError and code 351)
2018-10-26 10:50:37,542+09 INFO  [org.ovirt.engine.core.bll.CommandCompensator] (default task-20) [18e068c6] Command [id=4d44969a-e9ba-4860-82d3-7eb810fd35c0]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.profiles.DiskProfile; snapshot: 210c96eb-2af9-4292-ac7e-98e89614adf4.
2018-10-26 10:50:37,542+09 INFO  [org.ovirt.engine.core.bll.CommandCompensator] (default task-20) [18e068c6] Command [id=4d44969a-e9ba-4860-82d3-7eb810fd35c0]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StorageDomainDynamic; snapshot: 28ce0e8d-1073-4e82-99a4-0be72a508696.
2018-10-26 10:50:37,542+09 INFO  [org.ovirt.engine.core.bll.CommandCompensator] (default task-20) [18e068c6] Command [id=4d44969a-e9ba-4860-82d3-7eb810fd35c0]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StorageDomainStatic; snapshot: 28ce0e8d-1073-4e82-99a4-0be72a508696.
2018-10-26 10:50:37,548+09 ERROR [org.ovirt.engine.core.bll.storage.domain.AddPosixFsStorageDomainCommand] (default task-20) [18e068c6] Transaction rolled-back for command 'org.ovirt.engine.core.bll.storage.domain.AddPosixFsStorageDomainCommand'.
2018-10-26 10:50:37,553+09 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-20) [18e068c6] EVENT_ID: USER_ADD_STORAGE_DOMAIN_FAILED(957), Failed to add Storage Domain vmstore. (User: admin@internal-authz)

vdsm.log : 
2018-10-26 10:50:36,967+0900 INFO  (jsonrpc/0) [vdsm.api] START connectStorageServer(domType=6, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'id': u'00000000-0000-0000-0000-000000000000', u'connection': u'10.10.10.15@tcp:/lustre/vmstore', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'vfs_type': u'lustre', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.161.104,48628, flow_id=5f2e2f52-eab8-4c16-a5cd-727d492f855f, task_id=950eedf2-c5d2-4f70-a529-d1471e2f0db5 (api:46)
2018-10-26 10:50:36,969+0900 INFO  (jsonrpc/0) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore' (storageServer:167)
2018-10-26 10:50:36,969+0900 INFO  (jsonrpc/0) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore mode: None (fileUtils:197)
2018-10-26 10:50:36,969+0900 INFO  (jsonrpc/0) [storage.Mount] mounting 10.10.10.15@tcp:/lustre/vmstore at /rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore (mount:204)
2018-10-26 10:50:37,025+0900 INFO  (jsonrpc/0) [IOProcessClient] Starting client Global (__init__:308)
2018-10-26 10:50:37,036+0900 INFO  (ioprocess/240628) [IOProcess] Starting ioprocess (__init__:437)
2018-10-26 10:50:37,054+0900 INFO  (jsonrpc/0) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'00000000-0000-0000-0000-000000000000'}]} from=::ffff:192.168.161.104,48628, flow_id=5f2e2f52-eab8-4c16-a5cd-727d492f855f, task_id=950eedf2-c5d2-4f70-a529-d1471e2f0db5 (api:52)
2018-10-26 10:50:37,054+0900 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.09 seconds (__init__:573)
2018-10-26 10:50:37,094+0900 INFO  (jsonrpc/4) [api.host] START getAllVmStats() from=::ffff:192.168.161.104,48628 (api:46)
2018-10-26 10:50:37,096+0900 INFO  (jsonrpc/4) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:192.168.161.104,48628 (api:52)
2018-10-26 10:50:37,097+0900 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:573)
2018-10-26 10:50:37,143+0900 INFO  (jsonrpc/7) [vdsm.api] START connectStorageServer(domType=6, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'id': u'f8d72480-9c0e-43ce-9e3b-eeb253dbc5cf', u'connection': u'10.10.10.15@tcp:/lustre/vmstore', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'vfs_type': u'lustre', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.161.104,48628, flow_id=18e068c6, task_id=60973360-9013-415c-8c24-947f9b21bc8d (api:46)
2018-10-26 10:50:37,145+0900 INFO  (jsonrpc/7) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'f8d72480-9c0e-43ce-9e3b-eeb253dbc5cf'}]} from=::ffff:192.168.161.104,48628, flow_id=18e068c6, task_id=60973360-9013-415c-8c24-947f9b21bc8d (api:52)
2018-10-26 10:50:37,145+0900 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.00 seconds (__init__:573)
2018-10-26 10:50:37,149+0900 INFO  (jsonrpc/5) [vdsm.api] START createStorageDomain(storageType=6, sdUUID=u'28ce0e8d-1073-4e82-99a4-0be72a508696', domainName=u'vmstore', typeSpecificArg=u'10.10.10.15@tcp:/lustre/vmstore', domClass=1, domVersion=u'4', options=None) from=::ffff:192.168.161.104,48628, flow_id=18e068c6, task_id=21737927-1f37-4b1d-9ee9-9da08b68682f (api:46)
2018-10-26 10:50:37,359+0900 INFO  (itmap/0) [IOProcessClient] Starting client /glusterSD/host1.intel.dom:_engine (__init__:308)
2018-10-26 10:50:37,369+0900 INFO  (itmap/1) [IOProcessClient] Starting client /glusterSD/gluster1:vmdata (__init__:308)
2018-10-26 10:50:37,372+0900 INFO  (ioprocess/240646) [IOProcess] Starting ioprocess (__init__:437)
2018-10-26 10:50:37,380+0900 INFO  (itmap/2) [IOProcessClient] Starting client /glusterSD/host1.intel.dom:master (__init__:308)
2018-10-26 10:50:37,384+0900 INFO  (ioprocess/240653) [IOProcess] Starting ioprocess (__init__:437)
2018-10-26 10:50:37,392+0900 INFO  (ioprocess/240660) [IOProcess] Starting ioprocess (__init__:437)
2018-10-26 10:50:37,396+0900 INFO  (itmap/0) [IOProcessClient] Starting client /10.10.10.15:_lustre_vmstore (__init__:308)
2018-10-26 10:50:37,405+0900 INFO  (itmap/1) [IOProcessClient] Starting client /10.10.10.15@tcp:_lustre_vmstore (__init__:308)
2018-10-26 10:50:37,409+0900 INFO  (ioprocess/240670) [IOProcess] Starting ioprocess (__init__:437)
2018-10-26 10:50:37,417+0900 INFO  (itmap/2) [IOProcessClient] Starting client /host1.intel.dom:_iso (__init__:308)
2018-10-26 10:50:37,419+0900 INFO  (ioprocess/240679) [IOProcess] Starting ioprocess (__init__:437)
2018-10-26 10:50:37,428+0900 INFO  (ioprocess/240687) [IOProcess] Starting ioprocess (__init__:437)
2018-10-26 10:50:37,429+0900 INFO  (jsonrpc/5) [storage.StorageDomain] sdUUID=28ce0e8d-1073-4e82-99a4-0be72a508696 domainName=vmstore remotePath=10.10.10.15@tcp:/lustre/vmstore domClass=1 (nfsSD:70)
2018-10-26 10:50:37,447+0900 INFO  (jsonrpc/5) [IOProcessClient] Starting client 28ce0e8d-1073-4e82-99a4-0be72a508696 (__init__:308)
2018-10-26 10:50:37,458+0900 INFO  (ioprocess/240698) [IOProcess] Starting ioprocess (__init__:437)
2018-10-26 10:50:37,484+0900 INFO  (jsonrpc/5) [storage.StorageDomain] Creating domain metadata directory u'/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md' (fileSD:449)
2018-10-26 10:50:37,489+0900 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2018-10-26 10:50:37,492+0900 INFO  (jsonrpc/3) [vdsm.api] START repoStats(domains=[u'f5d75ae4-3f13-465d-99a5-2594e6d2e94c']) from=::1,54244, task_id=85721ebf-8318-4c08-82f0-cc3230076f13 (api:46)
2018-10-26 10:50:37,492+0900 INFO  (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'f5d75ae4-3f13-465d-99a5-2594e6d2e94c': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000182245', 'lastCheck': '8.1', 'valid': True}} from=::1,54244, task_id=85721ebf-8318-4c08-82f0-cc3230076f13 (api:52)
2018-10-26 10:50:37,492+0900 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:573)
2018-10-26 10:50:37,503+0900 INFO  (jsonrpc/5) [storage.xlease] Formatting index for lockspace u'28ce0e8d-1073-4e82-99a4-0be72a508696' (version=1) (xlease:653)
2018-10-26 10:50:37,531+0900 INFO  (jsonrpc/5) [vdsm.api] FINISH createStorageDomain error=Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.00103131 s, 0.0 kB/s\n" from=::ffff:192.168.161.104,48628, flow_id=18e068c6, task_id=21737927-1f37-4b1d-9ee9-9da08b68682f (api:50)
2018-10-26 10:50:37,532+0900 ERROR (jsonrpc/5) [storage.TaskManager.Task] (Task='21737927-1f37-4b1d-9ee9-9da08b68682f') Unexpected error (task:875)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run
    return fn(*args, **kargs)
  File "<string>", line 2, in createStorageDomain
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method
    ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2591, in createStorageDomain
    storageType, domVersion)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 87, in create
    remotePath, storageType, version)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 465, in _prepareMetadata
    cls.format_external_leases(sdUUID, xleases_path)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 1200, in format_external_leases
    xlease.format_index(lockspace, backend)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 661, in format_index
    index.dump(file)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 761, in dump
    file.pwrite(INDEX_BASE, self._buf)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 994, in pwrite
    self._run(args, data=buf[:])
  File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 1011, in _run
    raise cmdutils.Error(args, rc, "[suppressed]", err)
Error: Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.00103131 s, 0.0 kB/s\n"
2018-10-26 10:50:37,532+0900 INFO  (jsonrpc/5) [storage.TaskManager.Task] (Task='21737927-1f37-4b1d-9ee9-9da08b68682f') aborting: Task is aborted: u'Command [\'/usr/bin/dd\', \'iflag=fullblock\', u\'of=/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases\', \'oflag=direct,seek_bytes\', \'seek=1048576\', \'bs=256512\', \'count=1\', \'conv=notrunc,nocreat,fsync\'] failed with rc=1 out=\'[suppressed]\' err="/usr/bin/dd: error writing \'/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases\': Invalid argument\\n1+0 records in\\n0+0 records out\\n0 bytes (0 B) copied, 0.00103131 s, 0.0 kB/s\\n"' - code 100 (task:1181)
2018-10-26 10:50:37,532+0900 ERROR (jsonrpc/5) [storage.Dispatcher] FINISH createStorageDomain error=Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.00103131 s, 0.0 kB/s\n" (dispatcher:86)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 73, in wrapper
    result = ctask.prepare(func, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper
    return m(self, *a, **kw)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1189, in prepare
    raise self.error
Error: Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore/28ce0e8d-1073-4e82-99a4-0be72a508696/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.00103131 s, 0.0 kB/s\n"
2018-10-26 10:50:37,532+0900 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call StorageDomain.create failed (error 351) in 0.39 seconds (__init__:573)
2018-10-26 10:50:37,606+0900 INFO  (jsonrpc/2) [vdsm.api] START disconnectStorageServer(domType=6, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'id': u'f8d72480-9c0e-43ce-9e3b-eeb253dbc5cf', u'connection': u'10.10.10.15@tcp:/lustre/vmstore', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'vfs_type': u'lustre', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.161.104,48628, flow_id=82d6091f-a1c0-4304-9dec-30bc317d4182, task_id=a21c6614-aef7-46f2-84f9-6b96f6c97f4d (api:46)
2018-10-26 10:50:37,606+0900 INFO  (jsonrpc/2) [storage.Mount] unmounting /rhev/data-center/mnt/10.10.10.15@tcp:_lustre_vmstore (mount:212)


Actual results:
As above


Expected results:
Be able to add luster file system as posix compatible FS storage domain


Additional info:
1. luster is POSIX compatible FS.
2. If the direct option is present, bs must be a multiple of 4K (4096) (for 8K, 12K, 256K, 1M, 8M, etc.) for operation. For example, if you enter a value such as 512 or 4095, it will not work. The error message is as follows.

'error message: dd: error writing [filename]: invalid argument'

Comment 1 okok102928 2018-10-26 02:06:44 UTC
vfstype is lustre, not luster. I entered it wrong.

Comment 2 Tal Nisan 2018-10-28 14:54:53 UTC
Hi,
Is it a 4K block storage or a 512b?

Comment 3 okok102928 2018-10-29 04:26:41 UTC
(In reply to Tal Nisan from comment #2)
> Hi,
> Is it a 4K block storage or a 512b?

1. The sector size of the block device used in lustre FS is 4K (512e) sector, AF.
However, the block device of 512 byte sector has the same result.

2. I contacted the Principal architect at lustre FS. He said lustre FS client should work with a block size that is a multiple of 4K during the direct IO operation. He said it was not easy to modify the file system client.

3. luster FS backfilesystem: ldiskfs is formatted with 4k block size.

As you can see, I do not speak English well. Thank you for your understanding.
Please tell me if you need more information.

Comment 4 okok102928 2018-11-02 07:42:11 UTC
Is that the information you said?

Waiting for an answer.

Comment 5 Tal Nisan 2018-11-05 09:36:36 UTC
Yes, that was my question.
For now we do not support 4K block file system, this will change with bug 1592916 which will be released in 4.3, marking this bug as a duplicate and you can track the work on the original bug

*** This bug has been marked as a duplicate of bug 1592916 ***


Note You need to log in before you can comment on or make changes to this bug.