Local login hangs, but SSH succeeds

T

torsionbar28

Hi folks, I have a FreeBSD server that's been running reliably for years. I recently upgraded the OS and now I cannot login locally any more. Logging in via SSH works fine though. Logging in at the local console (ttyu1, no gui) it appears to accept the username and password ok, but then "hangs" before reaching the shell prompt. I've waited many minutes, but it never gets there. Issuing a Ctrl-C will kick it back to the login prompt.

It was on 12.4 until about a week ago. I upgraded it to 13.0, then 13.1, 13.2, and 14.0 all back to back one day. Patched up to date now, at version 14.0-RELEASE-p5. It did not have this problem on 12.4. There is no additional software installed, this machine is just an NFS server. Any ideas? Console output is below, as captured via IPMI (I changed the domain name to "mydomain" for this post). Note the "Ctrl-C" near the bottom. After the "You have new mail" is where it hangs, never displaying a shell prompt. Logging in via ssh as the same user works fine no issues. I'm stumped!

Code: 
FreeBSD/amd64 (multivac.int.mydomain.net) (ttyu1)

login: root
Password:
Last login: Thu Feb 22 21:28:06 on ttyu1
FreeBSD 14.0-RELEASE-p5 (GENERIC) #0: Tue Feb 13 23:37:36 UTC 2024

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List:        https://www.FreeBSD.org/lists/questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

To change this login announcement, see motd(5).
You have new mail.
^C
FreeBSD/amd64 (multivac.int.mydomain.net) (ttyu1)

login:

Thank you in advance for any ideas. This behavior occurs for all user accounts on the machine right now.
 
I was able to work around this problem by changing the user's shell. All users including root had /bin/sh as their shell. Changing this value to anything else like csh or bash, the problem goes away.


chsh -s /usr/local/bin/bash root


I'm curious though why /bin/sh does not work properly. Also, is there a way to define the default shell for future users added to the system, so I don't have to manually change the shell each time?
 
Then the problem is in one of the shell source files for sh, please check those. Changing the root shell to anything but sh will open up a complete other can of worms. Be warned.
 
Personally, I use bash for the root login. However, as others said above: if you do that, make sure you have other logins that can use /bin/sh or /bin/csh, for emergencies.
 
Crivens said:
Changing the root shell to anything but sh will open up a complete other can of worms. Be warned.
Click to expand...
This subject comes up from time to time.

When operating as root, and particularly under duress, I want the shell with which I am most familiar.

For nearly 30 years, I have used a privately curated copy of /bin/ksh (statically linked) as root's login shell:
Code: 
[gunsynd.684] $ file /bin/ksh
/bin/ksh: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), statically linked, for FreeBSD 13.2, FreeBSD-style, stripped
[gunsynd.685] $ grep ksh /etc/shells
/bin/ksh
/usr/local/bin/ksh
I have never found a good reason not to do this. I don't believe that root's login shell should impact anything other than interactive root logins (or using "su -"). I should add that nobody else has root access on my systems -- so no nasty surprises for colleagues.

I guess that there's no better time to ask if anyone knows of changes that were required when root's login shell changed from /bin/csh to /bin/sh in FreeBSD 14.0.
 
Crivens said:
Then the problem is in one of the shell source files for sh, please check those.
Click to expand...
Forgive my ignorance, I'm well versed in Linux, HP-UX, and Irix, but fairly new to FreeBSD. Could you elaborate on how to check the source files? When I do a freebsd-update, I see the following message "src component not installed, skipped", does that mean I don't have source files installed?

I also ran a 'freebsd-update IDS' but the only files it identified has having a mismatched hash were config files; no binaries.
 
torsionbar28 said:
Hi folks, I have a FreeBSD server that's been running reliably for years. I recently upgraded the OS and now I cannot login locally any more. Logging in via SSH works fine though. Logging in at the local console (ttyu1, no gui) it appears to accept the username and password ok, but then "hangs" before reaching the shell prompt. I've waited many minutes, but it never gets there. Issuing a Ctrl-C will kick it back to the login prompt.

It was on 12.4 until about a week ago. I upgraded it to 13.0, then 13.1, 13.2, and 14.0 all back to back one day. Patched up to date now, at version 14.0-RELEASE-p5. It did not have this problem on 12.4. There is no additional software installed, this machine is just an NFS server. Any ideas? Console output is below, as captured via IPMI (I changed the domain name to "mydomain" for this post). Note the "Ctrl-C" near the bottom. After the "You have new mail" is where it hangs, never displaying a shell prompt. Logging in via ssh as the same user works fine no issues. I'm stumped!

Code: 
FreeBSD/amd64 (multivac.int.mydomain.net) (ttyu1)

login: root
Password:
Last login: Thu Feb 22 21:28:06 on ttyu1
FreeBSD 14.0-RELEASE-p5 (GENERIC) #0: Tue Feb 13 23:37:36 UTC 2024

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List:        https://www.FreeBSD.org/lists/questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

To change this login announcement, see motd(5).
You have new mail.
^C
FreeBSD/amd64 (multivac.int.mydomain.net) (ttyu1)

login:

Thank you in advance for any ideas. This behavior occurs for all user accounts on the machine right now.
Click to expand...

The working ssh login, is that with root, too?
 
torsionbar28 said:
Forgive my ignorance, I'm well versed in Linux, HP-UX, and Irix, but fairly new to FreeBSD. Could you elaborate on how to check the source files? When I do a freebsd-update, I see the following message "src component not installed, skipped", does that mean I don't have source files installed?

I also ran a 'freebsd-update IDS' but the only files it identified has having a mismatched hash were config files; no binaries.
Click to expand...
I think when they wrote "source files", they didn't mean "the source code to the OS, applications, and the shell", but instead they meant "files that the shell sources when it starts". When a user logs in (meaning when they start a shell), that shell runs a variety of scripts automatically. Those include /etc/profile, all files in /etc/profile.d, the same in /usr/local/etc, and usually things like .profile and .shrc in the user's home directory. Those files could easily contain something that hangs when running locally. My advice would be to go in and disable all of them, by renaming them for example to /etc/NOT.profile and so on, and then trying. If that works, re-enable them one at a time.
 
To trace the execution of the various source files, you can add the following to the start of /etc/profile:
Code: 
exec 2>/tmp/trace.$$
set -x
Examination of the trace file will help pin-point the place where the problem is.

Make sure that you have a root shell running in another window so you can reverse the change without having to login!
 
cracauer@ said:
The working ssh login, is that with root, too?
Click to expand...
Yes, with root too. The behavior is consistent even with a newly created test account. I did an adduser and created an account called "testuser". Testuser can login fine via ssh, but the testuser local console login hangs before reaching the shell prompt, same as the other accounts.
 
ralphbsz said:
I think when they wrote "source files", they didn't mean "the source code to the OS, applications, and the shell", but instead they meant "files that the shell sources when it starts". When a user logs in (meaning when they start a shell), that shell runs a variety of scripts automatically. Those include /etc/profile, all files in /etc/profile.d, the same in /usr/local/etc, and usually things like .profile and .shrc in the user's home directory. Those files could easily contain something that hangs when running locally. My advice would be to go in and disable all of them, by renaming them for example to /etc/NOT.profile and so on, and then trying. If that works, re-enable them one at a time.
Click to expand...
Ah ha, thank for you clearing that up. Since even a newly created testuser has the same problem, the issue must be something in /etc/profile or /etc/profile.d I'm guessing, rather than something in a user's homedir. Since the problem is common to all users, even newly created ones. This gives me something to go on, thank you.
 
torsionbar28 said:
Ah ha, thank for you clearing that up. Since even a newly created testuser has the same problem, the issue must be something in /etc/profile or /etc/profile.d I'm guessing, rather than something in a user's homedir.
Click to expand...

Still doesn't explain why ssh login works. An interactive ssh login parses the same dotfile initialization files as a getty login.

What happens if you create a user with a csh login?
 
torsionbar28 said:
It was on 12.4 until about a week ago. I upgraded it to 13.0, then 13.1, 13.2, and 14.0 all back to back one day.
Click to expand...
Just an FYI for any future upgrades, you can upgrade to 14.0 directly from 12.4, there's no need to install any of the intermediate versions.

I suspect doing so may have left some files in a bad state, maybe there's still some merge issues in several files.
 
You must log in or register to reply here.
Back
Top