Encoding of Chinese Characters

In computer text applications, the GB encoding scheme most often renders simplified Chinese characters, while Big5 most often renders traditional characters. Although neither encoding has an explicit connection with a specific character set, the lack of a one-to-one mapping between the simplified and traditional sets established a de facto linkage.

Since simplified Chinese conflated many characters into one and since the initial version of the GB encoding scheme, known as GB2312-80, contained only one code point for each character, it is impossible to use GB2312 to map to the bigger set of traditional characters. It is theoretically possible to use Big5 code to map to the smaller set of simplified character glyphs, although there is little market for such a product. Newer and alternative forms of GB have support for traditional characters. In particular, mainland authorities have now established GB 18030 as the official encoding standard for use in all mainland software publications. The encoding contains all East Asian characters included in Unicode 3.0. As such, GB 18030 encoding contains both simplified and traditional characters found in Big-5 and GB, as well as all characters found in Japanese and Korean encodings.

Unicode deals with the issue of simplified and traditional characters as part of the project of Han unification by including code points for each. This was rendered necessary by the fact that the linkage between simplified characters and traditional characters is not one-to-one. While this means that a Unicode system can display both simplified and traditional characters, it also means that different localization files are needed for each type.

The Chinese characters used in modern Japanese have also undergone simplification, but generally to a lesser extent than with simplified Chinese, it's worth mentioning that Japanese writing system reduced the number of Chinese characters in daily use, which was also part of the Japanese language reforms, thus, a number of complex characters were written phonetically. Reconciling these different character sets in Unicode became part of the controversial process of Han unification. Not surprisingly, some of the Chinese characters used in Japan are neither 'traditional' nor 'simplified'. In this case, these characters cannot be found in traditional/simplified Chinese dictionaries.

As a conclusion, GB18030 is the better choice for both Simplified Chinese and Traditional Chinese. Accordingly, in Google's language settings, zh-Hans/lang_zh_Hans is for Simplified Chinese and zh-Hant/lang_zh_Hant is for Traditional Chinese.

http://en.wikipedia.org/wiki/Simplified_Chinese_characters
http://www.google.com/coop/docs/cse/resultsxml.html#chineseSearch

Set Language in Private Search Engine

Our previous posting Create Your Own Search Engine (http://koncordpartners.blogspot.com/2009/07/create-your-own-search-engine.html) could only display search results in English. Actually, the codes in that posting only have 4 parameters, you can add much more to suit your own needs. These 4 Google WebSearch Query Parameters are:

cx - Required. The cx parameter specifies a unique code that identifies a custom search engine. You must specify a Custom Search Engine using the cx parameter to retrieve search results from that CSE.

ie - Optional. The ie parameter sets the character encoding scheme that should be used to interpret the query string. The default ie value is latin1.

q - Optional. The q parameter specifies the search query entered by the user. Even though this parameter is optional, you must specify a value for at least one of the query parameters (as_epq, as_lq, as_oq, as_q, as_rq) to get search results.

sa - Required. It is indeed you submit the search query to Google search engine.

Today, we add three more parameters to show how the Simplified Chinese and Traditional Chinese can be interchangeable as search result.

hl - Optional. The hl parameter specifies the interface language (host language) of your user interface. To improve the performance and the quality of your search results, you are strongly encouraged to set this parameter explicitly. However, a successful search engine does not rely only on the parameters, but also settings of search engine itself. That is why you may also need your own account. “Site language”, is not only the site language, but also the search engine host language (the display language other than search result). Naturally, it should be set as English. “Your site encoding”, which is the searching language, should be GB18030 in this example.

lr - Optional. The lr (language restrict) parameter restricts search results to documents written in a particular language. Google WebSearch determines the language of a document by analyzing:

- the top-level domain (TLD) of the document's URL
- language meta tags within the document
- the primary language used in the body text of the document

c2coff - Optional. The c2coff parameter enables or disables the Simplified and Traditional Chinese Search feature. Please note, this parameter is only related to Chinese. For other language, no need to use this. The default value for this parameter is "0" (zero), meaning that the feature is enabled. Values for the c2coff parameter are: 1 is disabled, and 0 is enabled.

Following is codes with these two parameters. It means the search will be restricted to only websites wrote in three languages: US English, Simplified Chinese and Traditional Chinese.

<!DOCTYPE  html  PUBLIC  "-//W3C//DTD  XHTML  1.0  Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html  xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
        <head>
                <title>
                        This  is  private  search  engine  using  Google  facility
                </title>
        </head>
        <body>
                </br>
                </br>
                </br>
                </br>
                </br>
                </br>
                </br>
                </br>
                <form  id="cse-search-box"  action="http://www.google.com/cse">
                        <input  type="hidden"  value="partner-pub-3597878264183301:140hrt-vklq"  name="cx">
                        <input  type="hidden"  value="GB18030"  name="ie">
                        <input  type="hidden"  value="lang_en|lang_zh-Hans|lang_zh-Hant"  name="lr">
                        <input type="hidden" value="0" name="c2coff">
                        <p  align="center">
                                <img  alt="Google"  src="http://www.google.com/intl/en_us/images/logo.gif"  />
                        </p>
                        <p  align="center">
                                <input  size="50"  name="q">
                                <input  type="submit"  value="Search"  name="sa">
                        </p>
                </form>
                <p  align="center">
                          Koncord  Private  Search  Engine  Based  on  New  Google  Technology
                </p>
        </body>
</html>

http://www.google.com/coop/docs/cse/resultsxml.html