How does zfs handel torrents anyway? because I got poor read performance when reading from a torrent files compared to a file that was uploaded by eq samba?phoenix said:And rtorrent with pre-allocate space enabled will drag down a ZFS pool.
Matty said:How does zfs handel torrents anyway? because I got poor read performance when reading from a torrent files compared to a file that was uploaded by eq samba?
will all the small bits of a torrent be spread all over the harddisk and is this causing the poorer performance?
wonslung said:you shouldnt' have to patch rtorrent.
It's QUITE simple to just set rtorrent to a maximum amount of memory with a line line this:
Code:max_memory_usage = 1024M
void
MemoryChunk::unmap() {
if (!is_valid())
throw internal_error("MemoryChunk::unmap() called on an invalid object");
+ if (msync(m_ptr, m_end - m_ptr,MS_INVALIDATE) != 0)
+ throw internal_error("MemoryChunk::unmap() - msync() system call failed");
+
if (munmap(m_ptr, m_end - m_ptr) != 0)
throw internal_error("MemoryChunk::unmap() system call failed: " + std::string(rak::error_number::current().c_str()));
}
try http://forums.freebsd.org/showpost.php?p=63019&postcount=46tobiastheviking said:I am getting the problem with "Active" memory, not from rtorrent(which i do have, but there is no correlation between usage and memory usage).
During testing i have also disabled rtorrent and sabnzbd+, and just done a cp from a UFS disk to a ZFS disk(or from ZFS to ZFS for that matter), and gotten the problem.
As it is, right now, both rtorrent and sabnzbd+ are running(on a UFS drive), and i had <100M Active. Then i started an rsync of some 5+ gb data. Active memory is now 1160M. Neither rtorrent or sabnzbd+ is actually doing anything right now, they are just running.
Done and done.Matty said:
tobiastheviking said:Done and done.
Well, i currently have an uptime of 21 days, which is bound to be a record on this system.
I still have 1132M marked as Active, though i can't see what could possibly be using it. But for now, it looks usable.
garrettmoore said:I'm still having issues. I have 8GB of ram now and it's been good for a while, but I'm downloading a 52GB torrent and my write performance keeps going to hell. Running that perl command to flush memory works, but is both tedious and sloppy.
Has changing min/max vnodes been the fix for everyone else? When I tried lowering my maxvnodes to such a small amount my performance seemed to be much, much worse.
garrettmoore said:I'm still having issues. I have 8GB of ram now and it's been good for a while, but I'm downloading a 52GB torrent and my write performance keeps going to hell. Running that perl command to flush memory works, but is both tedious and sloppy.
Has changing min/max vnodes been the fix for everyone else? When I tried lowering my maxvnodes to such a small amount my performance seemed to be much, much worse.
xmanpsk said:Did you try rtorrent patch?
Matty said:Samba with sendfile and sharing a zfs filesystem? just a guess
wonslung said:I can vouch for the patch. I was a doubter myself having never really noticed the issue (i was on a high ram system with a lot of fast drives) but when i installed to a 2 gb ram machine it was clearly evident, even on ufs, that rtorrent eventually fills all the ram.
This patch fixes it. I've mailed it to the maintainer of rtorrent, i think it should be included.
The hilarious thing is that they insist that rtorrent is basically perfect and that it's just good at unconvering filesystem issues in different operating systems. This was from a developer comment to a single rtorrent bugreport confirmed by people using Linux, FreeBSD, MacOS X and OpenSolaris across multiple versions of reiserfs, ext2, ufs and zfs. The arrogance of rtorrent developers is mindboggling.wonslung said:i opened a bug report with rtorrent's people but thye insist the problem lies with FreeBSD's code, not rtorrent's.
Jago said:The hilarious thing is that they insist that rtorrent is basically perfect and that it's just good at unconvering filesystem issues in different operating systems. This was from a developer comment to a single rtorrent bugreport confirmed by people using Linux, FreeBSD, MacOS X and OpenSolaris across multiple versions of reiserfs, ext2, ufs and zfs. The arrogance of rtorrent developers is mindboggling.
garrettmoore said:I tried the patch supplied for libtorrent and I still have all of my memory eventually being marked as Inact when I download torrents. How can I check to make sure rtorrent is definitely using the version of libtorrent I modified and reinstalled?
portmaster -Rf rtorrent\*
or portupgrade -Rf rtorrent\*
.vfs.zfs.vdev.min_pending: 4
vfs.zfs.vdev.max_pending: 8
# zdb <pool>
ahci_load="YES"
aio_load="YES"
zfs_load="YES"
Vfs.zfs.prefetch_disable=1
socket options = TCP_NODELAY SO_SNDBUF=131072 SO_RCVBUF=131072
use sendfile = no
min receivefile size = 16384
aio read size = 16384
aio write size = 16384
aio write behind = yes