Description of problem: Running dbench -t 100 10 on fuse client of a distribute volume errors out when add-brick is performed.


Version-Release number of selected component (if applicable):
Mainline

How reproducible:
100%

Steps to Reproduce:
1. Create atleast 2 brick Distribute volume
2. fuse mount
3. run dbench -t 100 10 on the fuse client
4. add brick asap
  
Actual results:
dbench should run to completion

Expected results:
dbench fails

[root@sgowda dht]# dbench -t 100 10
dbench version 4.00 - Copyright Andrew Tridgell 1999-2004

Running for 100 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 20 secs
9 of 10 processes prepared for launch   0 sec
10 of 10 processes prepared for launch   0 sec
releasing clients
  10       444   155.97 MB/sec  warmup   1 sec  latency 94.509 ms
  10       458    80.41 MB/sec  warmup   2 sec  latency 1094.612 ms
  10       459    53.61 MB/sec  warmup   3 sec  latency 2088.226 ms
  10       459    40.21 MB/sec  warmup   4 sec  latency 2963.711 ms
  10       459    32.17 MB/sec  warmup   5 sec  latency 3963.891 ms
[511] open ./clients/client4/~dmtmp/SEED failed for handle 9999 (No such file or directory)
(512) ERROR: handle 9999 was not found
Child failed with status 1
[root@sgowda dht]# [501] open ./clients/client7/~dmtmp/ACCESS failed for handle 9997 (No such file or directory)
[501] open ./clients/client6/~dmtmp/ACCESS failed for handle 9997 (No such file or directory)
[429] open ./clients/client1/~dmtmp/PARADOX/STUDENTS.MB failed for handle 9988 (No such file or directory)
[405] open ./clients/client2/~dmtmp/PARADOX/STUDENTS.XG1 failed for handle 9985 (No such file or directory)
[485] open ./clients/client0/~dmtmp/PARADOX/STUDENTS.FSL failed for handle 9995 (No such file or directory)
[413] open ./clients/client8/~dmtmp/PARADOX/STUDENTS.YG1 failed for handle 9986 (No such file or directory)



Additional info:

LOGS:
[2013-04-09 07:50:41.542248] D [dht-common.c:269:dht_discover_cbk] 6-sng-dht: lookup of <gfid:4717484d-f18d-44a0-b136-02688d3d8624> on sng-client-9 returned error (No such file or directory)
[2013-04-09 07:50:41.542309] D [dht-layout.c:641:dht_layout_normalize] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(client3_3_lookup_cbk+0x491) [0x7f639a492a91] (-->/usr/local/lib/glusterfs/3git/x
lator/cluster/distribute.so(dht_discover_cbk+0x27b) [0x7f639a2499fb] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/distribute.so(dht_discover_complete+0x337) [0x7f639a2496b7]))) 6-sng-dht: path=<gfid:4717484d
-f18d-44a0-b136-02688d3d8624> err=No such file or directory on subvol=sng-client-9
[2013-04-09 07:50:41.543068] D [dht-common.c:269:dht_discover_cbk] 6-sng-dht: lookup of <gfid:4717484d-f18d-44a0-b136-02688d3d8624> on sng-client-9 returned error (No such file or directory)
[2013-04-09 07:50:41.543490] D [dht-layout.c:641:dht_layout_normalize] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(client3_3_lookup_cbk+0x491) [0x7f639a492a91] (-->/usr/local/lib/glusterfs/3git/x
lator/cluster/distribute.so(dht_discover_cbk+0x27b) [0x7f639a2499fb] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/distribute.so(dht_discover_complete+0x337) [0x7f639a2496b7]))) 6-sng-dht: path=<gfid:4717484d
-f18d-44a0-b136-02688d3d8624> err=No such file or directory on subvol=sng-client-9
[2013-04-09 07:50:41.543533] D [dht-hashfn.c:97:dht_hash_compute] 6-sng-dht: trying regex for PPTB1E4.TMP
[2013-04-09 07:50:41.544216] D [fuse-resolve.c:81:fuse_resolve_entry_cbk] 0-fuse: 4717484d-f18d-44a0-b136-02688d3d8624/PPTB1E4.TMP: failed to resolve (No such file or directory)
[2013-04-09 07:50:41.544797] D [dht-common.c:269:dht_discover_cbk] 6-sng-dht: lookup of <gfid:4717484d-f18d-44a0-b136-02688d3d8624> on sng-client-9 returned error (No such file or directory)
[2013-04-09 07:50:41.545045] D [dht-layout.c:641:dht_layout_normalize] (-->/usr/local/lib/glusterfs/3git/xlator/protocol/client.so(client3_3_lookup_cbk+0x491) [0x7f639a492a91] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/distribute.so(dht_discover_cbk+0x27b) [0x7f639a2499fb] (-->/usr/local/lib/glusterfs/3git/xlator/cluster/distribute.so(dht_discover_complete+0x337) [0x7f639a2496b7]))) 6-sng-dht: path=<gfid:4717484d-f18d-44a0-b136-02688d3d8624> err=No such file or directory on subvol=sng-client-9
[2013-04-09 07:50:41.545090] D [dht-hashfn.c:97:dht_hash_compute] 6-sng-dht: trying regex for PPTB1E4.TMP
[2013-04-09 07:50:41.546266] D [fuse-resolve.c:81:fuse_resolve_entry_cbk] 0-fuse: 4717484d-f18d-44a0-b136-02688d3d8624/PPTB1E4.TMP: failed to resolve (No such file or directory)