Bug 1521119
| Summary: | GlusterFS segmentation fault when deleting files from sharded tiered volume | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | vnosov <vnosov> | ||||||||
| Component: | tiering | Assignee: | bugs <bugs> | ||||||||
| Status: | CLOSED WONTFIX | QA Contact: | bugs <bugs> | ||||||||
| Severity: | urgent | Docs Contact: | |||||||||
| Priority: | unspecified | ||||||||||
| Version: | mainline | CC: | amukherj, bugs, jbyers, vnosov | ||||||||
| Target Milestone: | --- | ||||||||||
| Target Release: | --- | ||||||||||
| Hardware: | x86_64 | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2018-11-02 08:13:11 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
Created attachment 1363339 [details]
GlusterFS core file
Created attachment 1363340 [details]
GlusterFS log files
Sorry, it was FUSE mount on system where the volume is provisioned. This problem occurs as a side effect of the fix for:
Bug 1282390 - Data Tiering:delete command rm -rf not deleting
files the linkto file(hashed) which are under migration and
possible spit-brain observed and possible disk wastage.
tier_unlink() is freeing the xdata out from under the shards
causing the dict_*() segfaults. It looks like dict_unref()
should only be called when the dict_set_dynstr_with_alloc(,
DHT_IATT_IN_XDATA_KEY) path is taken.
Release 3.12 has been EOLd and this bug was still found to be in the NEW state, hence moving the version to mainline, to triage the same and take appropriate actions. Patch https://review.gluster.org/#/c/glusterfs/+/21331/ removes tier functionality from GlusterFS. https://bugzilla.redhat.com/show_bug.cgi?id=1642807 is used as the tracking bug for this. Recommendation is to convert your tier volume to regular volume (either replicate, ec, or plain distribute) with "tier detach" command before upgrade, and use backend features like dm-cache etc to utilize the caching from backend to provide better performance and functionality. |
Created attachment 1363338 [details] The volume glusterfs configuration Description of problem: Create tiered volume with 2099GB cold segment and 1GB hot segment. Enable sharding for the volume with features.shard-block-size: 16MB and cluster.tier-demote-frequency: 150. Enable CIFS export for the volume. Mount volume on Windows host. Write 100GB files to the volume. 3 files were written to the volume in the test. File deletion could be executed on the host or it could be executed on Linix system where the volume is provisioned and has SMB mount. In any case failure will be the same. In the test the volume was unmounted from the host and deletion was executed on SMB mount. After the deletion command was submitted GlusterFS hit segmentation fault. Version-Release number of selected component (if applicable): GlusterFS 3.12.3 How reproducible: 100% Steps to Reproduce: 1. See the attached text file. 2. 3. Actual results: One of GlusterFS processes hit segmentation fault. Expected results: The selected files have to be deleted as it is requested. Additional info: see GlusterFS core file, see GlusterFS log files.