Variants in Second-Level Names Registered in Top-Level Domains

Internationalized Domain Names for Applications (IDNA) allows host names in the DNS to contain characters from the Unicode repertoire. Some Unicode characters are considered to be "variants" of one another. Because of the 20th century reform of Chinese writing, there is often more than one representation of what Chinese speakers think of as the same character. Some languages written in Latin characters with accents and diacritical marks, known as decorated characters, allow the decorations to be omitted in some situations; for example, French sometimes omits accents on capital letters, depending on country and culture. Due to the difficulty of representing decorated characters in ASCII systems, many users have informally used undecorated characters in DNS host names, even when they are not linguistically equivalent to the decorated versions. There is no single agreed-on definition of "variant". In 2012, ICANN said that variants "occur when a single conceptual character can be identified with two or more different Unicode Code Points with graphic representations that may be visually similar" (this definition was previously available at http://www.icann.org/en/resources/idn/variant-tlds). ICANN's IDN Variant Issues Project report says that "[t]here is today no fully accepted definition for what may constitute a variant relationship between top-level labels". RFC 3743 (an Informational RFC, not the product of the IETF) says that the idea of variants is "wherein one conceptual character can be identified with several different Code Points in character sets for computer use". The proper handing of variant names has been a topic of extensive debate and research, with little consensus reached on how to handle them or even what characters are variants of each other. Many people would like variant names to behave "the same", for a diverse range of meanings of "same". In some cases, it is a textual similarity, such as variants having corresponding DNS records; in some, it is functional similarity, such as variant names resolving to the same web server; while in others, it is user experience similarity, such as names resolving to web sites that, while not identical, are perceived by human users as equivalent. This document provides a snapshot of variant handling in the top-level domains (TLDs) contracted by ICANN, so-called gTLDs (generic TLDs) and sTLDs (sponsored TLDs), as of late 2012. We chose those domains because ICANN requires each TLD to describe its IDN and variant practices, and the TLD zone files are available for inspection, to verify what actually goes into the zones. This document also contains a small sampling of so-called ccTLDs (country code TLDs, the TLDs that consist of two ASCII letters) for which we could find information. Since "variant" can mean vastly different things to different people, there is also no agreement about when two zones are supposed to "behave the same". Also, the gTLDs and sTLDs might have different views of what variants are and are not required to report to ICANN about their policies.

We use some terminology that has become generally agreed to when discussing variant names, although we openly admit that such agreement is not complete and the terminology continues to change. The IDN practices documents (see below) can identify sets of code points that are considered variants of each other using Language Variant Tables, defined in . A set of names in which the characters in each position are variants is known as a bundle or, more technically, as an "IDL Package". The variant rules vary among languages, and for the same language can vary among TLDs. Many languages do not define variant characters and hence do not have bundles. A name is allocated if sponsorship of that label in some zone has been granted. This is similar to what many people refer to as "registered". A name is active if it appears as an owner name in a zone. Most allocated names are active, but some are not. Some names cannot be registered at all. For example, some registries allow one name in a bundle to be registered and block the rest. Some names can only be allocated under certain conditions. For example, some registries permit only the registrant of one name in a bundle to register or activate other names in the same bundle. Multiple names in a bundle are provisioned in the TLD with identical NS records, so they all are handled by the same name servers. The DNAME DNS record creates a shadow tree of DNS records, roughly as though there were a CNAME in the shadow tree pointing to each name in the target tree. DNAMEs have been used both to provide resolution for several names in a bundle and to provide resolution for every name under a TLD.

ICANN has published a variety of documents on variant management. The most important are the "Guidelines for the Implementation of Internationalized Domain Names" issued in Version 1.0 and Version 3.0. ICANN says that TLDs are supposed to register an IDN practices document with IANA for each language and/or script in which the TLD accepts IDN registrations, to be entered in the IANA Repository of IDN Practices. The practices document lists the Unicode characters allowed in names in the language or script, which characters are considered equivalent, and which of an equivalent group is preferred. Some TLDs have been more diligent than others at keeping the registry up to date. Also, some TLDs have tables for a few languages and scripts, while others (notably .COM, .NET, and .NAME) have a large set of tables, including some for languages and scripts that are no longer spoken or used, such as Runic and Ogham. The authors also note that many of the tables in the IANA registry are clearly out of date, containing URLs of policy pages that no longer exist and contact information for people who have left the registry. Some of the ICANN agreements with each TLD describe the TLD's IDN practices, but most don't.

This list covers most of the current set of gTLDs. In most cases, the authors have also checked the zone files for the gTLD to verify or augment the policy description.

The .AERO TLD has no IDNs and no rules or practices for them.

The .ASIA domain accepts registrations in many Asian languages. They have IANA tables for Japanese, Korean, and Chinese. The IANA tables refer to their CJK IDN policies, which say that applied-for and preferred IDN variants are "active and included in the zone". No IDN publication mechanism is described in the documentation, but since the zone file contains no DNAMEs, they must be using parallel NS for variants.

ICANN gave the registry (Neustar) non-specific permission to register IDNs in a letter in 2004. The IDN rules were apparently discussed with ICANN but not defined; see Appendix 9 of the registry agreement. They have about a dozen IANA tables. No IDN publication mechanism is described, but from inspection, it appears that variants are blocked.

The IDN rules are described in Appendix S, Part VII.2 of the ICANN agreement. "Registry will take a very cautious approach in its IDN offerings. IDNs will be bundled with the equivalent ASCII domains". The only language is Catalan. No IDN publication mechanism is described. Appendix S includes "The list of non-ASCII-characters for Catalan language and their ASCII equivalent for the purposes of the defined service", which implicitly describes bundles. The bundles consist of names with accented and unaccented vowels, U+00E7 ("c with cedilla") and a plain c, and the Catalan "ela geminada" written as two l's separated by a U+00B7 ("middle dot") and the three characters "l-l". When a registrant registers an IDN, the registry also includes the ASCII version. From inspection of the zone file, the ASCII version is provisioned with NS, and the IDN is a DNAME alias of the ASCII version.

ICANN and Verisign have extensive correspondence about IDNs and variants, including letters to ICANN from Ben Turner and Russell Lewis. The IANA registry has tables for several dozen languages, including archaic languages such as hieroglyphics and Aramaic. Verisign publishes documents describing scripts and languages, character variants, registration rules, and additional registration logic. In Chinese, variants are blocked (see ). In other languages, there is no bundling or blocking.

The .COOP TLD has no IDNs and no rules or practices for them.

The IANA registry has a table for German. The German table notes that "the Eszet ... character used in the German script will be mapped to a double 's' string (i.e. 'ss')". The domain also offers names in Greek, Russian, Arabic, Korean, and other languages. The list and IDN tables are on the registry's web site. Afilias says (not in a published policy) that it does not allow Korean characters with different widths and that there are no variants in .INFO. Appendix 9 of the registry agreement refers to a 2003 letter from Paul Twomey that refers to blocking variants.

The .JOBS TLD has no IDNs and no rules or practices for them.

The zone file has about 22,000 IDNs. Afilias says (not in a published policy) that .MOBI supports Simplified Chinese only and that the language table for this is the same as that used by .CN. Variant characters are blocked from registration. The domain has no tables at IANA. Appendix S of the registry agreement, says that IDNs are provisioned according to .

The zone file has many IDNs, but spot checks find that many are lame or dead. A 2004 letter from Paul Twomey refers to . The registry has a detailed policy page. IDNs are accepted in Latin and Hebrew scripts, with plans for Arabic, Chinese, Japanese, Korean, Cyrillic, and Greek. They do no bundling or blocking, but names that may be confusable due to visual similarity are not allowed. These are apparently determined by manual inspection, which is practical due to the very small size of the domain.

The .NAME TLD is managed the same as .COM.

The .NET TLD is managed the same as .COM.

A 2003 letter from Paul Twomey refers to . The registry has a list of IDN languages, several written in Latin script, plus Chinese and Korean. A Questions page states that Chinese names have been accepted since January 2010 and Cyrillic names in seven languages since February 2011. The practices for some, but not all, of the Latin languages are registered with IANA. A Chinese language policy form on the Public Interest Registry (PIR) web site says that the ZH-CN and ZH-TW IDNs use the corresponding ccTLD tables from IANA, and check boxes say that Variant Registration Polices and Variant Management Policies are applicable but don't say what those policies are. Private correspondence describes not-yet-public rules for variants in Chinese and Cyrillic in .ORG that restrict the number of variants that a registration can have. The Korean language policy form says that it uses the KRNIC table for Korean from IANA and that there are no variants.

The .POST TLD appears to have no registrations at all yet.

The .PRO TLD has no IDNs and no rules or practices for them.

The zone has many IDNs. It is probably operating according to a 2004 letter from Paul Twomey to Neustar, which did not mention specific TLDs. Its policy page has links to IDN practices for 17 languages, all but one of which are registered with IANA. None of the Latin scripts do bundling or blocking. The Japanese practices say that variants are blocked. The Chinese practices document says: Therefore, in addition to the blocking mechanism, bundling is also implemented for the Chinese language IDNs. When registering a Chinese language IDN (primary domain name) up to two additional variant domain names will be automatically registered. The first variant will consist entirely of simplified Chinese characters that correspond to those comprising the primary domain name. The second variant will consist exclusively of traditional Chinese characters that correspond to those comprising the primary domain name. The primary domain name together with the requested variants constitutes a bundle on which all operations are atomic. For example, if the registrant adds a name server to the primary domain name, all names in the bundle will be associated with that new name server. The zone has no DNAME records, so the second paragraph strongly suggests parallel NS. The .TEL TLD, intended as an online directory, does not allow registrants to enter arbitrary Resource Records (RRs) in the zone. Nearly all names have NS records pointing to Telnic's own name servers. The A records all point to Telnic's own web server that shows directory information. NAPTR records provide telephone numbers of registrants if available. Users can only directly provision MX records. Currently, there are 16 domains, none of which are IDNs, that point to random other name servers and mostly appear to be parked.

The .TRAVEL TLD has no IDNs and no rules or practices for them.

The .XXX TLD has no IDNs and no rules or practices for them.

Some ccTLDs publish their IDN policies. This section is a non-exhaustive sampling of some of those policies. Note that few ccTLDs make their zone files available, so the authors could not validate the policies by looking in the zone files.

The .BG TLD (for Bulgaria) publishes a policy page. It has published an IDN table for the Bulgarian and Russian languages in . The policy does not mention variants.

The .BR TLD (for Brazil) publishes a policy page. It has published an IDN table for the Portuguese language in . Although the IDN table does not describe variants, the policy page says that bundles consist of names that are the same disregarding accents on vowels, cedillas on letter "c", and inserted or deleted hyphens. Only the registrant of a name in a bundle can register other names from the same bundle.

The .CL TLD (for Chile) publishes a policy page. It has published an IDN table for the Latin script in . The policy says that variants are not considered for registration.

The .CN TLD (for China) publishes its policy as . It has published an IDN table for the Chinese language in . The policy says that variants are "added into the zone file", presumably as NS records.

The .ES TLD (for Spain) publishes an IDN Area page. It allows ten accented vowels, U+00E7 ("c with cedilla"), U+00F1 ("n with tilde"), and the Catalan "ela geminada" written as two l's separated by a U+00B7 ("middle dot"). There are no published IDN tables, and there appears to be no variant policy.

The .EU TLD (for Europe) publishes a policy page. It has published IDN tables for three scripts in . There appears to be no variant policy.

The .GR TLD (for Greece) publishes a policy page and an FAQ. The policy says that all variants of a name under .GR are assigned to the domain owner, with the zone pointing the NS records of all the variants to the name server of the "main form" of the registered name. The FAQ says that domain names in Greek characters are inserted in the zone using their non-punctuated form in Punycode and that the punctuated form is associated with the non-punctuated with a DNAME record. It does not publish IDN tables in .

The .IL TLD (for Israel) publishes a policy page. It has published an IDN table for the Hebrew language in . There is no variant policy.

The .IR TLD (for Iran) publishes a policy page. It has published an IDN table for the Persian language in . The IDN table says that it will block registration of variants. However, the policy document says that no IDNs can be registered in .IR.

The .JP TLD (for Japan) publishes a policy page. It has published an IDN table for the Japanese language in . Each code point in that table defines no variants, which means there are no variants in registration or resolution.

The .KR TLD (for South Korea) appears to only publish its policy as an IDN table for the Korean language in . The policy in that table does not discuss variants.

The .MY TLD (for Malaysia) appears to only publish its policy as an IDN table for the Jawi language in ; however, IANA lists that as a table for "Malay (macrolanguage)". The policy in that table does not discuss variants.

The .NZ TLD (for New Zealand) publishes a policy page. It has published IDN tables for the Latin script in . The policy does not discuss variants.

The .PL TLD (for Poland) publishes a policy page. It has published IDN tables for numerous European languages in . The policy says that it will block registration of "look-alike" variants.

The .RS TLD (for Serbia) publishes a policy page. It has published IDN tables for the Serbian language, and the Latin script, in . The policy does not discuss variants.

The .RU TLD (for Russia) appears to only publish its policy as an IDN table for the Russian language in . The policy in that table does not discuss variants.

The .SA TLD (for Saudi Arabia) publishes a policy page. It has published an IDN table for the Arabic language in . The policy permits the registration of variants, but it is not clear whether others can register names with variants if the owner of a name has not registered them.

The .SE TLD (for Sweden) publishes a policy page. It has published IDN tables for the Swedish and Yiddish languages, and the Latin script, in . The policy does not discuss variants.

The .TW TLD (for Taiwan) appears to only publish its policy as an IDN table for the Chinese language in . The policy in that table does not discuss variants.

The .UA TLD (for Ukraine) publishes a policy page. It has published an IDN table for the Cyrillic script in . The policy does not discuss variants.

The .VE TLD (for Venezuela) appears to only publish its policy as an IDN table for the Spanish language in . The policy in that table does not discuss variants.

The .XN--90A3AC TLD (for Serbia) (U+0441 U+0440 U+0431) publishes a policy page. It has published IDN tables for the Cyrillic script in . The policy does not discuss variants.

The .XN--MGBERP4A5D4AR TLD (for Saudi Arabia) (U+0627 U+0644 U+0633 U+0639 U+0648 U+062F U+064A U+0629) appears to only publish its policy as an IDN table for the Arabic script in . The policy permits the registration of variants, but it is not clear whether others can register names with variants if the owner of a name has not registered them.

Many people contributed to this document, particularly Nacho Amadoz, Marc Blanchet, Michelle Coon, Jordi Iparraguirre, Frederico A. C. Neves, Vaggelis Segredakis, Doron Shikmoni, Andrew Sullivan, Dennis Tan, and Joseph Yee.

There are many potential security considerations for various methods of dealing with IDN variants. However, this document is only a catalog of current variant policies and does not address whether they are good or bad ideas from a security standpoint. The documents cited in the Terminology section have a little discussion of security considerations for IDN variants.