FreeBSD-11.0 possible issues with csh

It seem to be something wrong with the FreeBSD shell. I’m using FreeBSD-11.0r and this has never happen in previous versions of FreeBSD.

I ran this script:
Code:
#!/bin/csh
# ……….. RESTORE XP -  this works
echo "Restoring Windows-XP.  __ minutes.  Please wait ..."
umount -f /mnt/win/l
sleep 5
cat /mnt/win/r/win-xp/windows-xp.gz.* | gzip -dc | dd of=/dev/ada0s10 bs=64k
sleep 5
ntfs-3g /dev/ada0s10 /mnt/win/l
sleep 5
echo "All DONE!!!"
sleep 1200
exit

And this was the exact result:
Code:
Restoring Windows-XP.  __ minutes.  Please wait ...
312479+1 records in
312479+1 records out
20478657024 bytes transferred in 419.170100 secs (48855243 bytes/sec)
All DONExinit mate-session!

It took xinit mate-session right out of root/.history

---------------------------------
---------------------------------

Yesterday was my first time trying with FreeBSD-11.0 and this line is what I got at the end. I did not think to save it. I was in a state of shock, and figure I’ll save it if it happens again. It happen again as you see above. This is crazy! It leaked the entire command but the operation completed successful.
Code:
All DONEcat /mnt/win/r/win-10/windows-10.gz.* | gzip -dc | dd of=/dev/ada0s11 bs=64k!

This is a serious issue that could affect something elsewhere, but here it was harmless because both operations were successful. We got to catch these things early on.
 
!! in csh references the last command you ran from history. I used to type !! on the command line quite frequently to run the previous command; These days i just use the arrow keys (The way the default shell in FreeBSD only scrolls back through commands that match what you've already typed is so useful).

I ran the following test script on FreeBSD 9.2 and it did exactly the same thing.
Code:
#!/bin/csh

echo "Test!!!"
Code:
# csh test.sh
Testexit!

You should be able to just change the script to use /bin/sh as mentioned. I'm intrigued why there's a 20 minute wait at the end of the script before it exits though...
 
cat /mnt/win/r/win-xp/windows-xp.gz.* | gzip -dc | dd of=/dev/ada0s10 bs=64k

This is very dangerous. Install another drive, and ada0 is not the one you expected. Also, UUOC and writing scripts for csh(1). Using triple "shriek stops" did uncover a bug, so I'll not mention it otherwise.

The bug happens on 10-stable also.

Please enter a bug report at https://bugs.freebsd.org/bugzilla/ with the simple test usdmatt shows above.
 
Are you sure it's a bug? From what I get from csh(1), history substitution doesn't get turned off just by enclosing things in quotes; you've got to remember that '!' is a special character for csh and escape it with a backslash (\).
 
It seems like a bug, but I don't know. If !! was included as part of the script, it would make more sense, but just echoing it in output is questionable. If it was being expanded as part of a variable, even. But it was output, not input.
 
You really shouldn't use csh(1) for scripting. It has a couple of quirks, especially when it comes to redirection. Use sh(1) for scripts, use csh(1) for interactive use.
Hi SirDice; I learn the difference last winter when I created my own jail framework. It using some of what FreeBSD has to offer by itself, but it’s nothing to brag about. Here’s the story: I ended up with 8 custom tiny scripts. Would you believe I had to use csh for two of them because sh would not work? csh came to the rescue. And it goes the other way around too. I’ll post what I have someday soon for proof of concept. My new habit is use sh and if it don't work try csh, but don't give in to csh unless you have to. It better be important.

You should be able to just change the script to use /bin/sh as mentioned. I'm intrigued why there's a 20 minute wait at the end of the script before it exits though...
Hey Usdmatt; Thanks for putting two and two together and enlightening us of the underlying functionality of it all. The 20-minute sleep is so not to miss the message, the first time, while getting a cup of coffee :)

This is very dangerous. Install another drive, and ada0 is not the one you expected. Also, UUOC and writing scripts for csh(1). Using triple "shriek stops" did uncover a bug, so I'll not mention it otherwise.
Wblock, I see no UUOC. I be interested in testing your replacement if it does all of what I use can do. It will dd, gzip and split a 20GB partition or slice in under 7 minutes. It takes Linux 20 minutes and it will never fail.

[edited] I got to learn to keep it short.
 
cat /mnt/win/r/win-xp/windows-xp.gz.* | gzip -dc
can be replaced with
gzip -dc /mnt/win/r/win-xp/windows-xp.gz.*
I think this can also be replaced with
zcat /mnt/win/r/win-xp/windows-xp.gz.*

I would suggest testing the output partition with file(1) before overwriting it with dd(1). Also, look at using ntfsclone(8) from sysutils/fusefs-ntfs which could save a lot of time.
 
wblock@ I have lots of interest in your examples. I checked my notes and I been there long ago without success. I’m going to start a new thread so to leave this for farther investigation if needed. I want to get to the bottom of this gzip and dd thing for both partition and file usage. Thank you


Since usdmatt verified the flaw and ljboiler concluded why it happens, I figure it must have been in the base system long before FreeBSD touch and have never seen this flaw. If no one has file a report, maybe one of the developers who read this will decide if it should be fix or not. If it don’t hurt anything he might let it be. Until then there will be no more !! out of me.

[Edited] I got to start keeping it short.
 
!! as input is fine. This is doing it on output. The upstream csh or tcsh mailing list would probably be the place to ask, though.
 
Would you believe I had to use csh for two of them because sh would not work? csh came to the rescue. And it goes the other way around too. I’ll post what I have someday soon for proof of concept. My new habit is use sh and if it don't work try csh, but don't give in to csh unless you have to. It better be important.
I for one, would be interested in knowing what is not working in sh? I build my boxes without csh so I never get bit on the nose by it.
 
Sorry about the late reply leebrown66. I been reorganizing my HDD and testing my scripts so to be ready for the p2 or p1 REALEASE, and I messed up a lot just to see what would actually continue work flawlessly. This is from my hand-constructed jail. Even the kernel don’t know it exist ... or it just a fun thought for me.

I’m force to use csh here:
Code:
#!/bin/csh
# @@@@@@@@@@@@@@@@@@@@@@@@  KILL DEV

umount -A -t devfs	# kill dev
sleep 1
unsetenv JAIL_1		# kill env
sleep 1
exit
… and here:
Code:
#!/bin/csh
# @@@@@@@@@@@@@@@@@@@@@@@@@ VIEW COMMAND
#
cd $JAIL_1
printenv $JAIL_1 /mnt/e/Database1

# a chance to view the result. 
sleep 500
exit
... and maybe this. I forgot because I been at it with other things lately.
Code:
#!/bin/csh
# @@@@@@@@@@@@@@@@@@@@@@@@  START JAIL

#mount_nullfs -o /dev/ad4s3a /usr/ports /mnt/e/Database1/usr/ports
#mount_nullfs -o /dev/ad4s3a /usr/src /mnt/e/Database1/usr/src

setenv JAIL_1 /mnt/e/Database1
cd $JAIL_1
mount -t devfs devfs /mnt/e/Database1/dev
devfs -m /mnt/e/Database1/dev rule -s 4 applyset
devfs -m /mnt/e/Database1/dev rule apply path tun0 unhide

ln -s dev/null kernel
#touch $JAIL_1/etc/fstab

jail $JAIL_1 ss1.local 10.0.0.1 /bin/sh
I see that I accually use csh in 4 script out of 8. So basically other than the above, I don’t use csh except when dealing with the MBR and when dd’ing my FreeBSD primaries for backup reasons. This is by choice. Everything else usually get cp -p or tar. Soon I’ll be testing ntfsclone as I go.
 
OK, so presumably you are using JAIL_1 (in this example) as a persistent variable across different scripts, so script1 could set it, script2 use it and then script3 change it.

You are right in that csh is the only shell which will allow you to modify the parent's environment. What's usually done here is to write the data into a source-able script, ie:

t1.sh
Code:
#!/bin/sh
echo "JAIL_1=/mnt/e/Database1" > ./sh-vars.txt

t2.sh
Code:
#!/bin/sh
if [ -f ./sh-vars.txt ]; then
 . ./sh-vars.txt
 echo "JAIL_1 is $JAIL_1"
else
 echo "sh-vars.txt expected, not found"
fi

t3.sh
Code:
rm ./sh-vars.txt

result:

Code:
$ ./t1.sh
$ ./t2.sh
JAIL_1 is /mnt/e/Database1
$ ./t3.sh
$ ./t2.sh
sh-vars.txt expected, not found
 
OK, so presumably you are using JAIL_1 (in this example) as a persistent variable across different scripts, so script1 could set it, script2 use it and then script3 change it.

You are right in that csh is the only shell which will allow you to modify the parent's environment. What's usually done here is to write the data into a source-able script, ie:

t1.sh
Code:
#!/bin/sh
echo "JAIL_1=/mnt/e/Database1" > ./sh-vars.txt

t2.sh
Code:
#!/bin/sh
if [ -f ./sh-vars.txt ]; then
 . ./sh-vars.txt
 echo "JAIL_1 is $JAIL_1"
else
 echo "sh-vars.txt expected, not found"
fi

t3.sh
Code:
rm ./sh-vars.txt

result:

Code:
$ ./t1.sh
$ ./t2.sh
JAIL_1 is /mnt/e/Database1
$ ./t3.sh
$ ./t2.sh
sh-vars.txt expected, not found
The script below started my serious use of csh. Because it seems that when dealing with environment variables for jails built from scratch in the way I done exposed more truth. So presumably what we just notice about csh, I think the early developers of FreeBSD knew that ! and possibly other special characters usages would lead to danger for inexperienced users. They knew if a user uses one in a typo it could blow a system fuse, depending. I been there I think, I blew a few MBR’s or fuses, now I kind of see why. The only one that gave me a real clue was when I notice my ntfs partitions were gone. No matter how I tried to recover them I end up having to resort to my backup drive a day latter. Of course this could have been something else I done wrong. Anyway, I think they figure why not simply discourage usages, which only makes good sense it is the most important shell of the base system.

This is pure speculation but it makes sense and now I/we know how NOT to use it, if one ever do use it. If the kernel silently tells me sh don’t work, I’m going to try csh since I know it can be the solution sometimes.

A work around just to use sh could cause other problems down the road, but I’ll play with it just to see. However, I make no promise to give up what has been working perfectly before my eyes everyday no matter how hard I pound and play with it for nearly a year. I rip and tear down jails just to test the backup-tar for months at a time because I made changes daily and needed to keep up with them. I had nothing better to do while trying to learn webbing.

I usually do things by the handbook first, and then I read blogs for weeks that may point out what the handbook might be missing, but with respect for the handbook, where else could they/we have start. I like how-to and if it don’t work for whatever simple reason I usually find other ways that may not be so common. I use section-6 blogs and a few others where I had to mix and match, then modify a few things make it work. So don’t let what I done throw you off. I’m moving to vImages jails and when I’m done it will never fail (thanks to 11). Doing things the hard way makes you pay attention to every detail 100 times over, but it cost you major time.

leebrown66, your scripts are going to be my road map to real scripting. Once I see a relation to what I have interest in it makes understanding how things work much more clearer when nothing else did. I think it been the fear of going in too deep for me. I had other plans.

Anyway, in the past I made many changes to this file to make it work differently for something else but it should give you the complete idea of where those other scripts above came from and why I use accidently used csh for something else which brought me to posting the question. If someone uses it, step through it a few pieces at a time to catch any possible flaw do to any of my previous changes. If I remember correctly, it still 100% flawless. Try session-6, then you know why I came up with what I have. It even proves newer versions of FreeBSD causes minor changes. Still section6 been in like Flint with the best of the rest!

http://tutorials.section6.net/home

Database_1 is the name of the jail
JAIL_1 is the environment like $DESTDIR

For full effect - uncomment some of what is within the file.

A_InstallWorld - use this to re-install or make a new jail.

b_delete.jail - must delete old jail to re-install or use the backup tar.

c_untar.jail - using backup tar to replace jail, but leave slop, delete it first?

d_start - start the jail.

e_kill-environment - must kill env before closing the jail console.

tar-it - if you including new stuff, back it up with this... Will delete old tar first!

view-env - view your jail environment, see what is in that shell.

csh is cool here!

Code:
#!/bin/csh
# @@@@@@@@@@@@@@@@@@@@@@@@@@    Database1 InstallWorld
# @@@@@@@@@@@@@@@@@@@@@@@@@@    DELETE Database1 first
#  ..................................    Build RESORT  ... first 2 lines you can change
#  ..................................    Build RESORT  ... to make a new complete RESORT
# @@@@@@@@@@@@@@@@@@@@@@@@@@    DELETE Database1
# ..... >
#cd /mnt/e/
#chflags -R noschg /mnt/e/Database1/*
#sleep 3
#rm -Prf /mnt/e/Database1
#sleep 3
# ..... >
#cd /mnt/j/MAKE/
#rm -Pr Database1.tgz
#sleep 3
#
#
#mkdir /mnt/j/MAKE/Database1
#echo "--is error?--"
#echo "-----"
#echo "-----"
echo "Now deleting old.  We will InstallWorld in 4 minutes..."
#cd /mnt/j/MAKE/Database1
#chflags -R noschg *
#rm -Prf *
#rm Pf .cshrc
#rm Pf .profile  no no#
#cd ..
#  Never delete the first directory even if hand made.

# @@@@@@@@@@@@@@@@@@@@@@@@@@    InstallWorld
#sleep 2
#mkdir /mnt/j/MAKE/Database1
cd /mnt/e/
setenv JAIL_1 /mnt/e/Database1
# cp /etc/mnt/j/MAKE.conf /etc/mnt/j/MAKE.conf
mkdir -p $JAIL_1/dev
mkdir -p $JAIL_1/etc
mkdir -p $JAIL_1/usr/ports
mkdir -p $JAIL_1/usr/src
mkdir -p $JAIL_1/usr/tmp
chmod 777 $JAIL_1/usr/tmp
cd /usr/src/
# make buildworld
#make installworld DESTDIR=$JAIL_1
make installworld DESTDIR=$JAIL_1 SRCCONF=/etc/src.conf

# @@@@@@@@@@@@@@@@@@@@@@@@@@   mergemaster
sleep 2
mergemaster -a -C -D ${JAIL_1}
#mergemaster -i ${JAIL_1}
sleep 2
cd /usr/src/etc
cp /etc/resolv.conf $JAIL_1/etc
cp /etc/mnt/j/MAKE.conf $JAIL_1/etc

# @@@@@@@@@@@@@@@@@@@@@@@@@@    distribution
sleep 2
make distribution DESTDIR=$JAIL_1 OPTIONS_UNSET=OPENSSH OPTIONS_UNSET=OPENSSL
sleep 2
rm -Pf $JAIL_1/etc/ssl/openssl.cnf
mkdir -p $JAIL_1/usr/local/openssl
cp /etc/ssl/openssl.cnf $JAIL_1/etc/ssl
cd $JAIL_1/usr/local/openssl/
sleep 2
ln -s ../../../etc/ssl/openssl.cnf openssl.cnf
# ...............................
# ...............................
# ...............................
# ...............................

# @@@@@@@@@@@@@@@@@@@@@@@@@@    devfs
mount -t devfs devfs $JAIL_1/dev
devfs -m $JAIL_1/dev rule -s 4 applyset
sleep 2
cd $JAIL_1
ln -s dev/null kernel
touch $JAIL_1/etc/fstab

# @@@@@@@@@@@@@@@@@@@@@@@@@@    CLEANUP - TURN-OFF ENV
sleep 2
umount -A -t devfs
unsetenv JAIL_1
sleep 2

# @@@@@@@@@@@@@@@@@@@@@@@@@@    MAKE BACKUP tar afar
#cd /mnt/j/MAKE/
#rm -Pr Database1.tgz
#sleep 1
#cd /mnt/e
#tar cvzf /mnt/j/MAKE/Database1.tgz Database1/

# @@@@@@@@@@@@@@@@@@@@@@@@@@    delete temproot
#sleep 2
#cd /var/tmp/temproot
#chflags -R noschg *
#rm -Prf *
#rm P .cshrc
#rm P .profile
#cd ..
#rm -Prf temproot
#sleep 2
#exit

echo "-----";
echo "-- done ........................."
sleep 1000

No better than the rest, but no less than the best :)
 
Back
Top