| Summary: | server-side io-cache is preventing client-side io-cache from working | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | idadesub |
| Component: | io-cache | Assignee: | Raghavendra G <raghavendra> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |
| Severity: | low | Docs Contact: | |
| Priority: | low | ||
| Version: | mainline | CC: | gluster-bugs |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | --- | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Description
Raghavendra G
2009-11-12 17:06:09 UTC
Hi All! This is actually for gluster-3.0pre1, not the mainline. I'm excited to see the new io-cache working well with large caches. It's successfully caching a large 1gb block of data now, which is great. I did notice an odd interaction bug between server-side and client-side caches though. It seems that when I turn on server-side caching, the client will then always read the data from the server instead of from it's own cache. Here are my spec files: ------------------------------------------- server.vol: volume posix type storage/posix option directory /tmp/gluster end-volume volume posix-locks type features/locks subvolumes posix end-volume volume io-cache type performance/io-cache option cache-size 4gb subvolumes posix-locks end-volume volume io-threads type performance/io-threads subvolumes io-cache end-volume volume server type protocol/server subvolumes io-threads option transport-type tcp option auth.addr.io-threads.allow * end-volume ------------------------------------------- client.vol: volume client type protocol/client option transport-type tcp/client option remote-host server option remote-subvolume io-threads end-volume volume io-cache type performance/io-cache option cache-size 4gb subvolumes client end-volume ------------------------------------------- Watching the memory usage for the gluster process as I copy my 1gb test file around, I can see the server's memory usage climb to 1gb, which makes sense. I also see the memory usage on the client climb to 1gb. This goes at gigabit link speed. However, when I copy the same file a minute later, the client's memory usage drops to 0 and then rises to 1gb again. The copy also runs at link speed. And as a control, I also tried out this server.vol: ------------------------------------------- volume posix type storage/posix option directory /tmp/gluster end-volume volume posix-locks type features/locks subvolumes posix end-volume volume io-threads type performance/io-threads subvolumes posix-locks end-volume volume server type protocol/server option transport-type tcp option auth.addr.io-threads.allow * subvolumes io-threads end-volume ------------------------------------------- The first copy goes at link, and the second copy is read out of memory on the client, which is what I'd expect. So, I expect this means that the server-side cache is telling the client-side that the file's dirty, and so the client copies the data again. Still, this isn't that horrible of a bug, though, since it's still correct-but-slow behavior, and the kernel does a pretty good job of keeping files in memory without io-cache enabled on the server. Still, it'd be a nice thing to have working :) PATCH: http://patches.gluster.com/patch/2315 in master (performance/io-cache: don't use stat got in read_cbk if it is zero-filled.) I just tested the master branch and git, and it properly reads the file from the client cache :) Thanks for the quick fix! (In reply to comment #3) > I just tested the master branch and git, and it properly reads the file from > the client cache :) Thanks for the quick fix! you are welcome :) PATCH: http://patches.gluster.com/patch/2478 in release-2.0 (performance/io-cache: don't use stat got in read_cbk if it is zero-filled.) |