FreeBSD and UTF-8 Problems

Hello.

I've added
Code:
:charset=UTF-8:\
:lang=en_US.UTF-8:
to my login.conf file and now my locale is all set to "en_US.UTF-8".

However, I've noticed that the man pages are no longer displaying correctly and now that I started working with more files, I've also noticed that non-ASCII characters don't display correctly. What could be causing this?

When I do man ls, I see this.
Code:
LS(1)			FreeBSD General Commands Manual 		 LS(1)

NAME
     ls ∏Çö list directory contents

In my text files, vi is displaying letters in hex, I think. For example the word «búsquedas» is displayed as «b\xc3\xbasquedas» instead.

It seems like there is a character encoding conflict. I know I save my stuff using UTF-8 encoding without BOM outside of FreeBSD. My locale in FreeBSD is set to UTF-8. So I'm not sure what's going on here. Has anyone else experienced this?
 
I experienced this myself in a normal PuTTY SSH terminal, so it has nothing to do with syscons. I "fixed" it by aliasing man to man -o which forces the use of non localized man pages.
 
Is there another level of abstraction that I can add to get Unicode characters to display? I've heard something about ROXTerm and it being able to support Unicode, but honestly, I'm not even sure what that is or how (or if) it differs from SH.
 
xtaz said:
I experienced this myself in a normal PuTTY SSH terminal, so it has nothing to do with syscons. I "fixed" it by aliasing man to man -o which forces the use of non localized man pages.

I do it through PuTTY settings: Window -> Translation and choose UTF-8.
 
A couple of different things have to happen to have proper UTF-8 support on a terminal. First the application should know that it should output UTF-8 encoded strings, this is done with the locale(1) settings. Secondly the terminal program (the console on the machine or for example PuTTY running on remote machine) must know to interpret the input it gets from the remote end as UTF-8 and also have the proper fonts in Unicode encoding to be able to display the UTF-8 encoded Unicode characters.
 
It's got nothing to do with the PuTTY settings etc. The locale is properly set to UTF-8, PuTTY is also properly set to UTF-8. Everything else like typing Euro/Chinese characters etc. in text editors or my IRC client works perfectly fine. It's literally just that the man pages don't get formatted properly. Which as I said I "fixed" by stopping it trying to use localised man pages with the man -o command. Although this is something that I did several years ago. It may be fixed these days. I haven't actually tested it recently. In fact I've just unaliased the man -o alias and taken a look at several pages, and they all look fine. So it probably was fixed at some point. I'm currently running 10.0-BETA3.
 
Back
Top