Solved How to git clone the FreeBSD sources on a ZFS fs to the src dir.

Don ZioMario

Don ZioMario

Hello.

I'm using 14.0-RELEASE and I want to upgrade it from source code to 14.1-RELEASE. The problem is with the src directory. At the moment is not empty,but git clone does not want a full directory,it wants to create it by itself. This is what happens when I rename it :

Code: 
[root@marietto /usr]==> mv src src-old
mv: rename src to src-old: Operation not supported

I can't rename that directory with mv(1) when it's mounted, but I can zfs-rename(8) a dataset :

Code: 
[root@marietto /usr]==> zfs rename zroot3/usr/src zroot3/usr/src-old
[root@marietto /usr]==> git clone https://git.FreeBSD.org/src.git /usr/src

but I suspect that's wrong because in this way the source code will be stored in a directory,but shouldn't it goes into the "src" dataset ? But the src dataset is not empty. So what should I do ?

I tried to do something like this :

Code: 
[root@marietto /]==> cd /usr/src
[root@marietto /usr/src]==> mkdir src-old
[root@marietto /usr/src]==> mv * /usr/src-old
[root@marietto /usr/src]==> git clone https://git.FreeBSD.org/src.git /usr/src-new
[root@marietto /usr/src]==> cd /usr/src-new
[root@marietto /usr/src-new]==> mv * /usr/src
[root@marietto /usr/src-new]==> cd /usr/src
[root@marietto /usr/src]==> git checkout releng/14.1

Not sure at all that it is the correct procedure to follow in this case. I feel to be smart,but I'm sure that I'm not.
 
ok,but you made it with one only command what I did with seven ;D
A part this,does it means that the procedure is good ?
Do you know that if "git clone" finds /usr/src it refuses to work ? It wants to create it.
 
RhfUsQnPbC said:
You can git clone src in other directory, and mv the content of that src to /usr/src.
Click to expand...

Are you sure that your procedure is good ? because I tried,but when I run etcupdate -p or -B sometime it gives an error,sometimes it says nothing :

Code: 
[root@marietto /usr/src]==> etcupdate -p
[root@marietto /usr/src]==> etcupdate -B

I've performed another upgrade,this time on the UFS fs where I can do : "mv src src-old" and then "git clone ..." and in this case it works,etcupdate reports a lists of file,that I have attached.
 

Attachments

  • etc-141.zip
    129.4 KB · Views: 18
Title says ZFS. Is /usr/src a distinct ZFS dataset, mounted at /usr/src? If so then #7 is a way to do it. Another would be to use zfs commands to move the current dataset mountpoint, then create a new dataset and mount it at /usr/src.
 
cracauer@

Code: 
[root@marietto /usr/src]==> git clone https://git.FreeBSD.org/src.git /src.git

Clone in '/src.git' in corso...
remote: Enumerating objects: 4522472, done.
remote: Counting objects: 100% (383025/383025), done.
remote: Compressing objects: 100% (28638/28638), done.
remote: Total 4522472 (delta 377308), reused 354387 (delta 354387), pack-reused 4139447
Ricezione degli oggetti: 100% (4522472/4522472), 1.64 GiB | 606.00 KiB/s, fatto.
Risoluzione dei delta: 100% (3594149/3594149), fatto.
Aggiornamento dei file in corso: 100% (101087/101087), fatto.

[root@marietto /usr/src]==> mv src/* .
zsh: no matches found: src/*

maybe you mean :

Code: 
# mv /src.git/* .

but my /usr/src is full. your commands don't remove any file inside of it....

Code: 
[root@marietto /usr/src]==>  mv /src.git/* .

mv: rmdir ./.cirrus-ci: Directory not empty
mv: rmdir ./.git: Directory not empty
mv: rmdir ./.github: Directory not empty
mv: rmdir ./bin: Directory not empty
mv: rmdir ./cddl: Directory not empty
mv: rmdir ./contrib: Directory not empty
mv: rmdir ./crypto: Directory not empty
mv: rmdir ./etc: Directory not empty
mv: rmdir ./gnu: Directory not empty
mv: rmdir ./include: Directory not empty
mv: rmdir ./kerberos5: Directory not empty
mv: rmdir ./lib: Directory not empty
mv: rmdir ./libexec: Directory not empty
mv: rmdir ./release: Directory not empty
mv: rmdir ./rescue: Directory not empty
mv: rmdir ./sbin: Directory not empty
mv: rmdir ./secure: Directory not empty
mv: rmdir ./share: Directory not empty
mv: rmdir ./stand: Directory not empty
mv: rmdir ./sys: Directory not empty
mv: rmdir ./targets: Directory not empty
mv: rmdir ./tests: Directory not empty
mv: rmdir ./tools: Directory not empty
mv: rmdir ./usr.bin: Directory not empty
mv: rmdir ./usr.sbin: Directory not empty
 
Oh boy, making things complicated when it can be done easy.


UFS: rm -r /usr/src && git clone .... /usr/src

Same on ZFS. rm -r /usr/src will complain "Device busy", but it will empty /usr/src nevertheless, without removing the src directory itself.

rm(1) can't remove src dir on ZFS, it's the mount point of child dataset zroot/usr/src , only zfs-destroy(8) can remove.




RhfUsQnPbC said:
You can git clone src in other directory, and mv the content of that src to /usr/src.
Click to expand...
ziomario said:
Are you sure that your procedure is good ? because I tried,but when I run etcupdate -p or -B sometime it gives an error,sometimes it says nothing :
Click to expand...
Point etcupdate(8) to the non-default src directory with the -s option.

etcupdate -s /usr/src-new -p




I tend to build for testing purposed often on different branches. To avoid building the different branches over and over again, to save build time, I have following strategy:

On ZFS dataset zroot/usr/src ( /usr/src ) I have a full FreeBSD source clone, checked out for the current running system.

From that dateset a snapshot is taken and made a clone for different branches, e.g. /usr/src-14.1-STABLE , /usr/src-main. and checkout the respective branch. That way the objdir of every branch remains separated.

Additional WITH_META_MODE enabled building time is reduced signifantly even more.

The same can be done on UFS. A file tree cloning or mirroring utility like sysutils/cpdup or sysutils/clone can be used to duplicat /usr/src.
 
I've understood your method,but this is not different than mine,at post #1,so I know what will be the end of the story.
 
ziomario said:
cracauer@

Code: 
[root@marietto /usr/src]==> git clone https://git.FreeBSD.org/src.git /src.git

Clone in '/src.git' in corso...
remote: Enumerating objects: 4522472, done.
remote: Counting objects: 100% (383025/383025), done.
remote: Compressing objects: 100% (28638/28638), done.
remote: Total 4522472 (delta 377308), reused 354387 (delta 354387), pack-reused 4139447
Ricezione degli oggetti: 100% (4522472/4522472), 1.64 GiB | 606.00 KiB/s, fatto.
Risoluzione dei delta: 100% (3594149/3594149), fatto.
Aggiornamento dei file in corso: 100% (101087/101087), fatto.

[root@marietto /usr/src]==> mv src/* .
zsh: no matches found: src/*

maybe you mean :

Code: 
# mv /src.git/* .

but my /usr/src is full. your commands don't remove any file inside of it....

Code: 
[root@marietto /usr/src]==>  mv /src.git/* .

mv: rmdir ./.cirrus-ci: Directory not empty
mv: rmdir ./.git: Directory not empty
mv: rmdir ./.github: Directory not empty
mv: rmdir ./bin: Directory not empty
mv: rmdir ./cddl: Directory not empty
mv: rmdir ./contrib: Directory not empty
mv: rmdir ./crypto: Directory not empty
mv: rmdir ./etc: Directory not empty
mv: rmdir ./gnu: Directory not empty
mv: rmdir ./include: Directory not empty
mv: rmdir ./kerberos5: Directory not empty
mv: rmdir ./lib: Directory not empty
mv: rmdir ./libexec: Directory not empty
mv: rmdir ./release: Directory not empty
mv: rmdir ./rescue: Directory not empty
mv: rmdir ./sbin: Directory not empty
mv: rmdir ./secure: Directory not empty
mv: rmdir ./share: Directory not empty
mv: rmdir ./stand: Directory not empty
mv: rmdir ./sys: Directory not empty
mv: rmdir ./targets: Directory not empty
mv: rmdir ./tests: Directory not empty
mv: rmdir ./tools: Directory not empty
mv: rmdir ./usr.bin: Directory not empty
mv: rmdir ./usr.sbin: Directory not empty
Click to expand...

I assumed you start from an empty /usr/src

the checkout is just
git clone https://git.freebsd.org/src.git
Click to expand...
 

Let's go to the crux of the matter. When I give :

Code: 
[root@marietto /usr/src]==> etcupdate -p
[root@marietto /usr/src]==> etcupdate -B

can be good if I don't have any file list or if I get the error :

​

No previous tree to compare against, a sane comparison is not possible ?​


I'm trying to understand the origin of this situation that it does not seem sane.

I found a similar situation here :

etcupdate warning: "No previous tree to compare against, a sane comparison is not possible."

groups.google.com groups.google.com

he suggested to do :

# etcupdate extract
# etcupdate -p
# cd /usr/src
# make installworld
# etcupdate -B

but it also didn't work.
 
to install ports with git,
i know this is about src but the steps are pretty much the same

switch to root

Code: 
su

destroy the zfs dataset and recreate it

Code: 
zfs destroy -R zroot/usr/ports
zfs create zroot/usr/ports
zfs set mountpoint=/usr/ports zroot/usr/ports

checkout quarterly branch

Code: 
git clone --depth 1 https://git.FreeBSD.org/ports.git -b 2024Q2 /usr/ports
 

In my case,with a little of variations :

Code: 
# zfs rename zroot3/usr/src zroot3/usr/src-old
# zfs create zroot3/usr/src
# zfs set mountpoint=/usr/src zroot3/usr/src
# git clone --depth 1 https://git.FreeBSD.org/src.git /usr/src
# cd /usr/src
# git checkout releng/14.1
# make -j8 buildworld
# make -j8 kernel
# shutdown -r now
# cd /usr/src
# etcupdate -p
# make installworld
# etcupdate -B
# shutdown -r now
# cd /usr/src
# etcupdate resolve

I'm starting thinking that this is not the reason of the errors that I get and that I showed on post #13. Maybe it is only a coincidence that, when I can remove the src dir, I will not get that error but when I can't,I get it.
 
ziomario said:
In my case :

Code: 
# zfs destroy -R zroot3/usr/src
# zfs create zroot3/usr/src
# zfs set mountpoint=/usr/src zroot3/usr/src

it is not different than previous ones. I mean,git clone will find /usr/src and it will refuse to put there the code.
This brings me back to the previous methods suggested before. I'm starting to think that this is not the reason of the errors that I get and that I showed on post #13. Maybe it is only a coincidence that, when I can remove the src dir, I will not get that error but when I can't,I get it.
Click to expand...
That does indeed look strange. What does ls -la /usr/src say? What about
zfs destroy -R zroot3/usr/src
zfs create -o mountpoint=/usr/src zroot3/usr/src
ls -la /usr/src
If that does only give . and .., what about git clone https://git.FreeBSD.org/src.git /usr/src then? In short, zfs create shouldn't create additional files or subdirectories, and git clone shouldn't complain if the target directory is indeed empty.
 
I'm trying to upgrade 14.0-RELEASE to 14.1-RELEASE from source code :

Code: 
# zfs rename zroot3/usr/src zroot3/usr/src-old
# zfs create zroot3/usr/src
# cd /usr
# zfs set mountpoint=/usr/src zroot3/usr/src
# git clone --depth 1 https://git.FreeBSD.org/src.git /usr/src

Clone in '/usr/src' in corso...
remote: Enumerating objects: 104698, done.
remote: Counting objects: 100% (104698/104698), done.
remote: Compressing objects: 100% (88918/88918), done.
remote: Total 104698 (delta 22194), reused 42965 (delta 11850), pack-reused 0 (from 0)
Ricezione degli oggetti: 100% (104698/104698), 344.71 MiB | 7.87 MiB/s, fatto.
Risoluzione dei delta: 100% (22194/22194), fatto.
Aggiornamento dei file in corso: 100% (101087/101087), fatto.

# cd src
# git checkout releng/14.1

error: The path specifier 'releng/14.1' does not match any file known to git

fix : remove --depth 1 from the git command.
 
First, releng/13.4 doesn't seem to exist yet. Second, the --depth 1 messes with you here. So either do git clone --depth 1 --branch releng/13.3 ..., or omit the --depth flag. In both case git pull should get you the references to releng/13.4 once the branch gets created.
 
git checkout -b releng/14.1 origin/releng/14.1

Or at the initial checkout and to make it even more efficient, assuming you're going for minimal download / disk usage:

git clone --single-branch -b releng/14.1 --depth 1 https://git.FreeBSD.org/src.git /usr/src
 
You must log in or register to reply here.
Back
Top