tmpfs | /dev/shm | md questions

Chris_H

Chris_H

Greetings,
I'm not sure this is the best location for this. But here goes...
I'm attempting to better use the tools available to make systems more efficient. One way seems to me, is the use of /dev/shm. However, I can't find anything in the FreeBSD man() pages related to it. The closest thing I can find, is tmpfs(). Which isn't quite the same (as I understand it). I thought, given it's name, that it might be related to sysctl() kern.IPC.shm* -- the System V Inter Process Communication mem properties. Which is another question, for another thread. ;)
Anyway. My thoughts on these different implementations, is for using it for (web)browser caches, /tmp slices, and other things that will benefit from memory mapped space, and for easier "house keeping" of things usually stored in /tmp, or /var/tmp.
Anyone else using, or know more about /dev/shm, or the likes? Any thoughts?

Thank you for all your time, and consideration.

--Chris
 
As far as I know /dev/shm is a typical Linux thing, you won't find it on FreeBSD. Although it's named the similarly, it has nothing to do with System V IPC (shared memory). The System V shared memory is supported on FreeBSD by using options SYSVSHM in the kernel config (GENERIC has it). However, applications need to be specifically built to use it.

I'm quite fond of tmpfs(5), I use it mounted on /tmp/ for fast access to temporary files. Simply add this to /etc/fstab:
Code: 
tmpfs                   /tmp    tmpfs   rw,mode=1777    0       0
 
Ah, I think this is the difference between the Linux and FreeBSD implementation (from shm_open()):
Code: 
The functions were	reimplemented as system	calls using shared memory
     objects directly rather than files	in FreeBSD 7.0.

I'm guessing Linux uses files, hence the /dev/shm device?
 
http://www.cyberciti.biz/tips/what-is-d ... usage.html

tmpfs 3.9G 168K 3.9G 1% /dev/shm
Click to expand...

So it would seem that /dev/shm is just a tmpfs ... ?

http://stackoverflow.com/questions/9745 ... e-on-linux
/dev/shm is intended for a very special purpose, not for files to be put to by arbitrary programs.

In contrast, /tmp is exactly made for this. On my systems, /tmp is a tmpfs as well, in contrast to /var/tmp which is designed for putting larger files, potentially staying longer.
Click to expand...

So there's no real different, apparently...
 
SirDice said:
As far as I know /dev/shm is a typical Linux thing, you won't find it on FreeBSD. Although it's named the similarly, it has nothing to do with System V IPC (shared memory).
Click to expand...
Thank you for the clarification. While I'm getting better, where kern.IPC.shm* is concerned, after having to tweak some of their sizes, so Qt4 applications work correctly. I've still got a long ways to go. I'd give my eye teeth to get my hands on a printed copy of the POSIX spec. :)
SirDice said:
The System V shared memory is supported on FreeBSD by using options SYSVSHM in the kernel config (GENERIC has it). However, applications need to be specifically built to use it.

I'm quite fond of tmpfs(5), I use it mounted on /tmp/ for fast access to temporary files. Simply add this to /etc/fstab:
Code: 
tmpfs                   /tmp    tmpfs   rw,mode=1777    0       0
Click to expand...
I saw references to that, and read the man() pages on tmpfs(). In the absence of /dev/shm, it appears that's the way to go, on FreeBSD. Seems it'd be possible to symlink browser caches to locations within a tmpfs() mounted /tmp slice, no?

Thanks for the tips, @SirDice, and your thoughtful repl(y|ies). :)

--Chris
 
Last edited by a moderator:
Carpetsmoker said:
http://www.cyberciti.biz/tips/what-is-devshm-and-its-practical-usage.html

tmpfs 3.9G 168K 3.9G 1% /dev/shm
Click to expand...

So it would seem that /dev/shm is just a tmpfs ... ?
Click to expand...
Yea. When I first attempted to investigate the possibilities, the first link I found was this:
http://sharadchhetri.com/2012/08/15/devshm-mount-devshm/
which indicates almost all the same information.

Carpetsmoker said:
http://stackoverflow.com/questions/9745281/tmp-vs-dev-shm-for-temp-file-storage-on-linux
/dev/shm is intended for a very special purpose, not for files to be put to by arbitrary programs.

In contrast, /tmp is exactly made for this. On my systems, /tmp is a tmpfs as well, in contrast to /var/tmp which is designed for putting larger files, potentially staying longer.
Click to expand...
So there's no real different, apparently...
Click to expand...

Well. Given my intent is to improve performance, by mapping some data into memory based cache, in a simple fashion. It looks like the earlier pointers by @SirDice regarding the mount of /tmp via tmpfs(), or in perhaps a clever implementation via shm_open(), is the way to go.

This all assumes, of course, that I understand it all correctly. :)

Thanks to all of you, for your time, and thoughtful replies.

--Chris
 
Last edited by a moderator:
SirDice said:
As far as I know /dev/shm is a typical Linux thing, you won't find it on FreeBSD. Although it's named the similarly, it has nothing to do with System V IPC (shared memory). The System V shared memory is supported on FreeBSD by using options SYSVSHM in the kernel config (GENERIC has it). However, applications need to be specifically built to use it.

I'm quite fond of tmpfs(5), I use it mounted on /tmp/ for fast access to temporary files. Simply add this to /etc/fstab:
Code: 
tmpfs                   /tmp    tmpfs   rw,mode=1777    0       0
Click to expand...
I had to break from my earlier investigation regarding all this. But I now have another opportunity to try all this again.
I'm now wondering if there's any appreciable difference/advantage to tmpfs(), over say md() / mdmfs()? Any thoughts? If I should just jump on your approach, @SirDice, and go with tmpfs(). Any "tuning" I should consider? Really, I just want to achieve better speed where I can, with things I can have use it, and for easier "housekeeping" -- everything get's obliterated, on umount(), or rebooting.

Thanks.

--Chris
 
Last edited by a moderator:
SirDice said:
Code: 
tmpfs                   /tmp    tmpfs   rw,mode=1777    0       0
Click to expand...
@@SirDice
Isn't /tmp traditionally created with the following permissions?
Code: 
tmpfs                   /tmp    tmpfs   rw,mode=1776    0       0
Note the trailing 6, as opposed to the 7 in yours.

Thanks.

--Chris
 
Last edited by a moderator:
Chris_H said:
I'm now wondering if there's any appreciable difference/advantage to tmpfs(), over say md() / mdmfs()?
Click to expand...

Yes, there's a huge difference. mdconfig(8) and mdmfs(8) are static devices. tmpfs(5) is dynamically-sized, with an optional size limit. Better yet, the space comes from virtual memory. So if you run a program that needs RAM, the data in tmpfs(5) can be swapped out to disk.

It's an elegant solution. If you can find it, there was a Sun white paper on it. Oracle has since removed it, and I have not been able to find it again.
 
wblock@ said:
Chris_H said:
I'm now wondering if there's any appreciable difference/advantage to tmpfs(), over say md() / mdmfs()?
Click to expand...

Yes, there's a huge difference. mdconfig(8) and mdmfs(8) are static devices. tmpfs(5) is dynamically-sized, with an optional size limit. Better yet, the space comes from virtual memory. So if you run a program that needs RAM, the data in tmpfs(5) can be swapped out to disk.
Click to expand...
Indeed. That makes things quite flexible, which could sure be handy. But I suspect speed might be better, where md()* managed mount points are concerned. Experimentation is in order, on my part, I suspect.

wblock@ said:
It's an elegant solution. If you can find it, there was a Sun white paper on it. Oracle has since removed it, and I have not been able to find it again.
Click to expand...
Yes! This one really %#!ses me off!. Sun (Microsystems) provided volumes of useful/valuable data online, and Oracle has subsequently removed the bulk of it -- sort of makes Oracle an oxymoron. Doesn't it? As a result, I find myself with fewer, and fewer reasons to go there. I don't know what kind of marketing strategy they're employing. But seems to me, that a great many perspective clients, won't be exposed to their products, as a result.

I don't suppose you still have the original link to the paper?

Thank you very much for the reply, @wblock@!

--Chris
 
Last edited by a moderator:
Chris_H said:
Indeed. That makes things quite flexible, which could sure be handy. But I suspect speed might be better, where md* managed mount points are concerned. Experimentation is in order, on my part, I suspect.
Click to expand...
Hmm... After reading the document you kindly provided, @wblock@;
RAM disks use memory inefficiently; file data exists twice in both RAM disk memory and kernel memory,
and RAM disk memory that is not being used by the file system is wasted. RAM disk memory is main-
tained separately from kernel memory, so that multiple memory-to-memory copies are needed to update
file system data.
Click to expand...
It appears little, if any further "experimentation" on my part, is even worth considering. Unless the way FreeBSD creates/handles md() backed files/systems, is different enough to tinker with.

Thanks again, for the paper, @wblock@.

--Chris
 
Last edited by a moderator:
You must log in or register to reply here.
Back
Top