Serkan Çoban
2016-Mar-18 05:34 UTC
[Gluster-users] nfs-ganesha/pnfs read/write path on EC volume
I am using 2.3. But I can test 2.4 if there are centos packages.. Or I can try to build rpm packages from latest ganesha if it is easy to build.. On Fri, Mar 18, 2016 at 7:03 AM, Jiffin Tony Thottan <jthottan at redhat.com> wrote:> > > On 17/03/16 23:17, Serkan ?oban wrote: >> >> Hi Jiffin, >> Will these patches land in 3.7.9? > > > Hi Serkan, > > I moved all changes to ganesha [1] and got merged upstream (ganesha > V2.4-dev-9), > I missed to back port it to 2.3, so which ganesha build are u using? > > [1] https://review.gerrithub.io/#/c/263180/ > > Jiffin > >> Serkan >> >> On Tue, Feb 16, 2016 at 2:47 PM, Jiffin Tony Thottan >> <jthottan at redhat.com> wrote: >>> >>> Hi Serkan, >>> >>> I had moved out previous gfapi-side to ganesha and include all those >>> change >>> in single patch https://review.gerrithub.io/#/c/263180/ >>> >>> I will try get it reviewed and merge the patch as soon as possible. >>> >>> With Regards, >>> Jiffin >>> >>> On 14/02/16 21:54, Serkan ?oban wrote: >>>> >>>> Thanks for the answer, >>>> AFAIK, when using pNFS, every different file read/write should go to >>>> different server in order to utilize all servers in parallel. >>>> I am waiting for the patches and future releases.Thanks for all your >>>> efforts. >>>> >>>> Serkan >>>> >>>> On Wed, Feb 10, 2016 at 2:21 PM, Jiffin Tony Thottan >>>> <jthottan at redhat.com> wrote: >>>>> >>>>> Hi, >>>>> >>>>> Sorry for the delayed delayed response >>>>> >>>>> >>>>> On 10/02/16 13:51, Pranith Kumar Karampuri wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 02/10/2016 01:15 PM, Serkan ?oban wrote: >>>>>>> >>>>>>> Hi Jiffin, >>>>>>> >>>>>>> Any update about the write path? >>>>>> >>>>>> I saw him send some mails related to this, yesterday and day before. >>>>>> You >>>>>> will hear from him soon. >>>>>> >>>>>> Pranith >>>>>>> >>>>>>> >>>>>>> Serkan >>>>>>> >>>>>>> On Sun, Jan 31, 2016 at 5:00 PM, Jiffin Tony Thottan >>>>>>> <jthottan at redhat.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 31/01/16 16:19, Serkan ?oban wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> I am testing nfs-ganesha with pNFS on EC volume and I want to ask >>>>>>>>> some >>>>>>>>> questions. >>>>>>>>> Assume we have two clients: c1,c2 >>>>>>>>> and six servers with one 4+2 EC volume constructed as below: >>>>>>>>> >>>>>>>>> gluster volume create vol1 disperse 6 redundancy 2 >>>>>>>>> server{1..6}:/brick/b1 >>>>>>>>> \ >>>>>>>>> >>>>>>>>> server{1..6}:/brick/b2 \ >>>>>>>>> >>>>>>>>> server{1..6}:/brick/b3 \ >>>>>>>>> >>>>>>>>> server{1..6}:/brick/b4 \ >>>>>>>>> >>>>>>>>> server{1..6}:/brick/b5 \ >>>>>>>>> >>>>>>>>> server{1..6}:/brick/b6 >>>>>>>>> vol1 is mounted on both clients as server1:/vol1 >>>>>>>>> >>>>>>>>> Here is first question: When I write file1 from client1 and file2 >>>>>>>>> from >>>>>>>>> client2; which servers get the files? In my opinion server1 gets >>>>>>>>> file1 >>>>>>>>> and server2 gets file2 and do EC calculations and distribute chunks >>>>>>>>> to >>>>>>>>> other servers. Am I right? >>>>>>>>> >>>>>>>>> Can anyone explain detailed read/write path with pNFS and EC >>>>>>>>> volumes? >>>>>>>> >>>>>>>> >>>>> Currently in pNFS cluster , request is send to the first DS available >>>>> in >>>>> the >>>>> list. >>>>> So I am to planning to distribute different files among the available >>>>> DS, >>>>> as >>>>> a first step >>>>> I had send out patch in gluster (http://review.gluster.org/#/c/13402/) >>>>> After this one got merged, there are certain changes in ganesha side >>>>> too. >>>>> I had tested both changes in my setup and it was working as accepted. >>>>> >>>>> Once again thanks for pointing out this issue >>>>> >>>>> -- >>>>> Regards, >>>>> Jiffin >>>>> >>>>> >>>>> >>>>> >>>>>>>> I never tried pNFS with EC volume, will try the same by my own and >>>>>>>> reply >>>>>>>> to >>>>>>>> your question as soon as possible. >>>>>>>> -- >>>>>>>> Jiffin >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serkan >>>>>>>>> _______________________________________________ >>>>>>>>> Gluster-users mailing list >>>>>>>>> Gluster-users at gluster.org >>>>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Gluster-users mailing list >>>>>>>> Gluster-users at gluster.org >>>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Gluster-users mailing list >>>>>>> Gluster-users at gluster.org >>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users >>>>>> >>>>>> >
Jiffin Tony Thottan
2016-Mar-18 06:22 UTC
[Gluster-users] nfs-ganesha/pnfs read/write path on EC volume
On 18/03/16 11:04, Serkan ?oban wrote:> I am using 2.3. But I can test 2.4 if there are centos packages.. > Or I can try to build rpm packages from latest ganesha if it is easy to build..Unfortunately I can't find any epel/centos[1] packages for 2.4, its only for fedora24[2] packages CCing Kaleb who knows about next 2.3.x update. Following are the steps used in ci.centos.org to build rpms(I hope it works) # install NFS-Ganesha build dependencies yum -y install git bison flex cmake gcc-c++ libacl-devel krb5-devel dbus-devel libnfsidmap-devel libwbclient-devel libcap-devel libblkid-devel rpm-build redhat-rpm-config # install the latest version of gluster yum -y install centos-release-gluster yum -y install glusterfs-api-devel git init nfs-ganesha cd nfs-ganesha git fetch https://review.gerrithub.io/ffilz/nfs-ganesha next git checkout -b next FETCH_HEAD # update libntirpc git submodule update --init || git submodule sync mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Maintainer ../src && make rpm [1] http://mirror.centos.org/centos-7/7.2.1511/storage/x86_64/gluster-3.7/ [2] http://arm.koji.fedoraproject.org/koji/packageinfo?packageID=17239 -- Jiffin> On Fri, Mar 18, 2016 at 7:03 AM, Jiffin Tony Thottan > <jthottan at redhat.com> wrote: >> >> On 17/03/16 23:17, Serkan ?oban wrote: >>> Hi Jiffin, >>> Will these patches land in 3.7.9? >> >> Hi Serkan, >> >> I moved all changes to ganesha [1] and got merged upstream (ganesha >> V2.4-dev-9), >> I missed to back port it to 2.3, so which ganesha build are u using? >> >> [1] https://review.gerrithub.io/#/c/263180/ >> >> Jiffin >> >>> Serkan >>> >>> On Tue, Feb 16, 2016 at 2:47 PM, Jiffin Tony Thottan >>> <jthottan at redhat.com> wrote: >>>> Hi Serkan, >>>> >>>> I had moved out previous gfapi-side to ganesha and include all those >>>> change >>>> in single patch https://review.gerrithub.io/#/c/263180/ >>>> >>>> I will try get it reviewed and merge the patch as soon as possible. >>>> >>>> With Regards, >>>> Jiffin >>>> >>>> On 14/02/16 21:54, Serkan ?oban wrote: >>>>> Thanks for the answer, >>>>> AFAIK, when using pNFS, every different file read/write should go to >>>>> different server in order to utilize all servers in parallel. >>>>> I am waiting for the patches and future releases.Thanks for all your >>>>> efforts. >>>>> >>>>> Serkan >>>>> >>>>> On Wed, Feb 10, 2016 at 2:21 PM, Jiffin Tony Thottan >>>>> <jthottan at redhat.com> wrote: >>>>>> Hi, >>>>>> >>>>>> Sorry for the delayed delayed response >>>>>> >>>>>> >>>>>> On 10/02/16 13:51, Pranith Kumar Karampuri wrote: >>>>>>> >>>>>>> >>>>>>> On 02/10/2016 01:15 PM, Serkan ?oban wrote: >>>>>>>> Hi Jiffin, >>>>>>>> >>>>>>>> Any update about the write path? >>>>>>> I saw him send some mails related to this, yesterday and day before. >>>>>>> You >>>>>>> will hear from him soon. >>>>>>> >>>>>>> Pranith >>>>>>>> >>>>>>>> Serkan >>>>>>>> >>>>>>>> On Sun, Jan 31, 2016 at 5:00 PM, Jiffin Tony Thottan >>>>>>>> <jthottan at redhat.com> wrote: >>>>>>>>> >>>>>>>>> On 31/01/16 16:19, Serkan ?oban wrote: >>>>>>>>>> Hi, >>>>>>>>>> I am testing nfs-ganesha with pNFS on EC volume and I want to ask >>>>>>>>>> some >>>>>>>>>> questions. >>>>>>>>>> Assume we have two clients: c1,c2 >>>>>>>>>> and six servers with one 4+2 EC volume constructed as below: >>>>>>>>>> >>>>>>>>>> gluster volume create vol1 disperse 6 redundancy 2 >>>>>>>>>> server{1..6}:/brick/b1 >>>>>>>>>> \ >>>>>>>>>> >>>>>>>>>> server{1..6}:/brick/b2 \ >>>>>>>>>> >>>>>>>>>> server{1..6}:/brick/b3 \ >>>>>>>>>> >>>>>>>>>> server{1..6}:/brick/b4 \ >>>>>>>>>> >>>>>>>>>> server{1..6}:/brick/b5 \ >>>>>>>>>> >>>>>>>>>> server{1..6}:/brick/b6 >>>>>>>>>> vol1 is mounted on both clients as server1:/vol1 >>>>>>>>>> >>>>>>>>>> Here is first question: When I write file1 from client1 and file2 >>>>>>>>>> from >>>>>>>>>> client2; which servers get the files? In my opinion server1 gets >>>>>>>>>> file1 >>>>>>>>>> and server2 gets file2 and do EC calculations and distribute chunks >>>>>>>>>> to >>>>>>>>>> other servers. Am I right? >>>>>>>>>> >>>>>>>>>> Can anyone explain detailed read/write path with pNFS and EC >>>>>>>>>> volumes? >>>>>>>>> >>>>>> Currently in pNFS cluster , request is send to the first DS available >>>>>> in >>>>>> the >>>>>> list. >>>>>> So I am to planning to distribute different files among the available >>>>>> DS, >>>>>> as >>>>>> a first step >>>>>> I had send out patch in gluster (http://review.gluster.org/#/c/13402/) >>>>>> After this one got merged, there are certain changes in ganesha side >>>>>> too. >>>>>> I had tested both changes in my setup and it was working as accepted. >>>>>> >>>>>> Once again thanks for pointing out this issue >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Jiffin >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>>> I never tried pNFS with EC volume, will try the same by my own and >>>>>>>>> reply >>>>>>>>> to >>>>>>>>> your question as soon as possible. >>>>>>>>> -- >>>>>>>>> Jiffin >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serkan >>>>>>>>>> _______________________________________________ >>>>>>>>>> Gluster-users mailing list >>>>>>>>>> Gluster-users at gluster.org >>>>>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Gluster-users mailing list >>>>>>>>> Gluster-users at gluster.org >>>>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users >>>>>>>> _______________________________________________ >>>>>>>> Gluster-users mailing list >>>>>>>> Gluster-users at gluster.org >>>>>>>> http://www.gluster.org/mailman/listinfo/gluster-users >>>>>>>