软件一般采用三种方式来决定文本的字符集和编码:

检测文件头标识,提示用户选择,根据一定的规则猜测

最标准的途径是检测文本最开头的几个字节,开头字节Charset/encoding,如下表:

EF BB BF UTF-8

FE FF UTF-16/UCS-2, little endian

FF FE UTF-16/UCS-2, big endian

FF FE 00 00 UTF-32/UCS-4, little endian.

00 00 FE FF UTF-32/UCS-4, big-endian.