On Thu, Jul 21, 2016 at 12:23:01PM -0500, Russell R Poyner wrote:> One more data point for comparison > > I installed the stock samba 4.2 rpm on a centos 7 machine and ran > the same diskspd tests against a share configured with: > vfs objects = aio_pthread > aio read size = 1024 > aio read size = 1024 > > smb2 leases = yes > > I get 27MB/s with 4k blocks and 145MB/s with 64k blocks. Disabling > cacheing by passing the -h switch to diskspd lowered these to 72MB/s > and 11MB/s. Which I view as 'close enough' to wire speed. Thus it > seems that the dismal performance I see is associated with the > FreeBSD implementation somehow.That's interesting, but I'm afraid I don't know FreeBSD well enough to help here. This does imply the problem isn't Samba specific though (unless it's a complex interaction between Samba+FreeBSD).
Jeremy, I think this is exactly a complex interaction between FreeBSD and Samba. Best guess would be some system call that is fast in linux but slow in FreeBSD holding things back. Russ On 07/21/2016 01:00 PM, Jeremy Allison wrote:> On Thu, Jul 21, 2016 at 12:23:01PM -0500, Russell R Poyner wrote: >> One more data point for comparison >> >> I installed the stock samba 4.2 rpm on a centos 7 machine and ran >> the same diskspd tests against a share configured with: >> vfs objects = aio_pthread >> aio read size = 1024 >> aio read size = 1024 >> >> smb2 leases = yes >> >> I get 27MB/s with 4k blocks and 145MB/s with 64k blocks. Disabling >> cacheing by passing the -h switch to diskspd lowered these to 72MB/s >> and 11MB/s. Which I view as 'close enough' to wire speed. Thus it >> seems that the dismal performance I see is associated with the >> FreeBSD implementation somehow. > That's interesting, but I'm afraid I don't know FreeBSD well > enough to help here. This does imply the problem isn't Samba > specific though (unless it's a complex interaction between > Samba+FreeBSD).
Am 21.07.2016 um 20:56 schrieb Russell R Poyner:> Jeremy, > > I think this is exactly a complex interaction between FreeBSD and > Samba. Best guess would be some system call that is fast in linux but > slow in FreeBSD holding things back. > > Russ > > On 07/21/2016 01:00 PM, Jeremy Allison wrote: >> On Thu, Jul 21, 2016 at 12:23:01PM -0500, Russell R Poyner wrote: >>> One more data point for comparison >>> >>> I installed the stock samba 4.2 rpm on a centos 7 machine and ran >>> the same diskspd tests against a share configured with: >>> vfs objects = aio_pthread >>> aio read size = 1024 >>> aio read size = 1024 >>> >>> smb2 leases = yes >>> >>> I get 27MB/s with 4k blocks and 145MB/s with 64k blocks. Disabling >>> cacheing by passing the -h switch to diskspd lowered these to 72MB/s >>> and 11MB/s. Which I view as 'close enough' to wire speed. Thus it >>> seems that the dismal performance I see is associated with the >>> FreeBSD implementation somehow. >> That's interesting, but I'm afraid I don't know FreeBSD well >> enough to help here. This does imply the problem isn't Samba >> specific though (unless it's a complex interaction between >> Samba+FreeBSD). > >On my debian jessie server with zfs these settings seem to work. max xmit = 65536 socket options = TCP_NODELAY Copying an file from server to a windows 7 client increases from 50% to 80% network utilisation on an 1GB link without jumbo frames when i add these settings.
Thanks for all the zfs tuning tips. The point is a good one. I was concerned that zfs performance might be limiting, and posted tests run against a ufs formatted ram disk for that reason. The tests with the ram disk are slightly faster than the zfs backed tests, but still slower than tests run with samba on linux using xfs on a single hard disk. Russ Poyner
On Fri, Jul 22, 2016 at 08:30:42AM -0500, Russell R Poyner wrote:> > Thanks for all the zfs tuning tips. The point is a good one. > > I was concerned that zfs performance might be limiting, and posted tests run > against a ufs formatted ram disk for that reason. The tests with the ram > disk are slightly faster than the zfs backed tests, but still slower than > tests run with samba on linux using xfs on a single hard disk.Just a random thought: Purely for testing purposes, not for production use: Try "strict locking = no". If this improves things, then the upcoming robust shared mutexes will help you. Volker
On Fri, Jul 22, 2016 at 08:30:42AM -0500, Russell R Poyner wrote:> > Thanks for all the zfs tuning tips. The point is a good one. > > I was concerned that zfs performance might be limiting, and posted > tests run against a ufs formatted ram disk for that reason. The > tests with the ram disk are slightly faster than the zfs backed > tests, but still slower than tests run with samba on linux using xfs > on a single hard disk.Maybe a networking issue ? Once you've elimated disk bottlenecks there aren't many other places left to look.