FreeBSD internal databases.

Phishfry · Jul 12, 2022

I would like to learn about the internal database used in FreeBSD.
cap_mkdb(1)

I know FreeBSD pkg(7) uses a database. Is the above using the same database or separate?
The manpage has no details and I always wondered about blindly updating this db upon running commands following some webpage. So is there a user database or is just passwords? I don't care about the gory details just rough outline.
pwd_mkdb(8)
What is db(3) style? Is that Berkeley db?

What are the FreeBSD databases inside? pkg uses sqlite I think with contents in /var/cache/pkg.
So here we have pwd and cap databases.
Is that about it?
Edit# I see termcap(5) too.

zirias@ · Jul 12, 2022

db(3) is Berkeley db indeed, a very old version (I think 1.85 or something like that?). So, what you have here is simple key-value storage.

T-Daemon · Jul 12, 2022

Code:

# file /etc/login.conf.db 
/etc/login.conf.db: Berkeley DB 1.85 (Hash, version 2, native byte-order)

Phishfry · Jul 12, 2022

So there is no db engine? Just plain text formatted in a db style?
Whereas pkg indeed uses a database proper?

zirias@ · Jul 12, 2022

What's a "database proper"? There is a "db engine". Berkeley DB. On FreeBSD, the code is actually integrated in libc (/lib/libc.so.7). And it definitely isn't "plain text", it's a binary format for storing key-value pairs.

You could say BSDs used "nosql" before it was cool (again).

covacat · Jul 12, 2022

pkg uses sqlite3

Phishfry · Jul 12, 2022

OK I have to ask. Why? Is there something missing with db?
Not good with large amounts of data? Too many fields?

Why do you think they chose to include a db 'proper'.
You know what I mean. Not a library but a frameworks. sqlite.
Just like Zirias mentions. db is built into the C library and C is quickest right?
What are Berkeley DB pitfalls in this case?

Phishfry · Jul 12, 2022

When I build NanoBSD with packages the first time I run pkg the database is created (my guess).
The df -h size grows from 125MB to 145MB. So pkg database eats 20MB. Not huge but compared to db not small..

jbo@ · Jul 12, 2022

Zirias said:
On FreeBSD, the code is actually integrated in libc (/lib/libc.so.7).

That seems like an odd (read: "interesting") choice. How did this come to be?

zirias@ · Jul 12, 2022

jbodenmann said:
That seems like an odd (read: "interesting") choice. How did this come to be?

BSD tradition? A BSD system comes with db(3) functionality. It's not uncommon to have much more in libc than just standard C functions, e.g. all the POSIX stuff (which e.g. GNU's glibc has as well). I don't want to know what weird functions MSVCRT.DLL hides ?

Phishfry said:
[pkg, sqlite] OK I have to ask. Why? Is there something missing with db?
Not good with large amounts of data? Too many fields?

Schemas and indices*. pkg needs a lot more than a single key to access data. Of course, you could implement this on top of BDB, but why should you? Use the right tool for the job. There's probably no strong need for an SQL database, but it doesn't hurt either.

Phishfry said:
db is built into the C library and C is quickest right?

IIRC, sqlite is implemented in C as well.

*) edit: not sure this is relevant for pkg, but: db(3) is not thread-safe. Newer BDB versions are, AFAIK. Also, there's no such thing as transactions.