Main Page   Class Hierarchy   Alphabetical List   Compound List   File List   Header Files   Compound Members   File Members  

Transliterator Class Reference

Transliterator is an abstract class that transliterates text from one format to another. More...

#include <translit.h>

Class diagram for Transliterator:

UnicodeToHexTransliterator RuleBasedTransliterator NullTransliterator JamoHangulTransliterator HexToUnicodeTransliterator HangulJamoTransliterator CompoundTransliterator

List of all members.


Public Members

enum  Direction { FORWARD, REVERSE }
Direction constant indicating the direction in a transliterator, e.g., the forward or reverse rules of a RuleBasedTransliterator. More...

virtual ~Transliterator ()
Destructor. More...

virtual Transliterator* clone () const
Implements Cloneable. More...

virtual int32_t transliterate (Replaceable& text, int32_t start, int32_t limit) const
Transliterates a segment of a string, with optional filtering. More...

virtual void transliterate (Replaceable& text) const
Transliterates an entire string in place. More...

virtual void transliterate (Replaceable& text, Position& index, const UnicodeString& insertion, UErrorCode& status) const
Transliterates the portion of the text buffer that can be transliterated unambiguosly after new text has been inserted, typically as a result of a keyboard event. More...

virtual void transliterate (Replaceable& text, Position& index, UChar insertion, UErrorCode& status) const
Transliterates the portion of the text buffer that can be transliterated unambiguosly after a new character has been inserted, typically as a result of a keyboard event. More...

virtual void transliterate (Replaceable& text, Position& index, UErrorCode& status) const
Transliterates the portion of the text buffer that can be transliterated unambiguosly. More...

virtual void finishTransliteration (Replaceable& text, Position& index) const
Finishes any pending transliterations that were waiting for more characters. More...

int32_t getMaximumContextLength (void) const
Returns the length of the longest context required by this transliterator. More...

virtual const UnicodeStringgetID (void) const
Returns a programmatic identifier for this transliterator. More...

virtual const UnicodeFiltergetFilter (void) const
Returns the filter used by this transliterator, or null if this transliterator uses no filter. More...

virtual void adoptFilter (UnicodeFilter* adoptedFilter)
Changes the filter used by this transliterator. More...

Transliterator* createInverse (void) const
Returns this transliterator's inverse. More...


Static Public Members

UnicodeStringgetDisplayName (const UnicodeString& ID, UnicodeString& result)
Returns a name for this transliterator that is appropriate for display to the user in the default locale. More...

UnicodeStringgetDisplayName (const UnicodeString& ID, const Locale& inLocale, UnicodeString& result)
Returns a name for this transliterator that is appropriate for display to the user in the given locale. More...

Transliterator* createInstance (const UnicodeString& ID, Direction dir = FORWARD)
Returns a Transliterator object given its ID. More...

void registerInstance (Transliterator* adoptedObj, UErrorCode& status)
Registers a instance obj of a subclass of Transliterator with the system. More...

void unregister (const UnicodeString& ID)
Unregisters a transliterator or class. More...

int32_t countAvailableIDs (void)
Return the number of IDs currently registered with the system. More...

const UnicodeStringgetAvailableID (int32_t index)
Return the index-th available ID. More...


Protected Members

 Transliterator (const UnicodeString& ID, UnicodeFilter* adoptedFilter)
Default constructor. More...

 Transliterator (const Transliterator&)
Copy constructor.

Transliterator& operator= (const Transliterator&)
Assignment operator.

virtual void handleTransliterate (Replaceable& text, Position& index, bool_t incremental) const = 0
Abstract method that concrete subclasses define to implement keyboard transliteration. More...

void setMaximumContextLength (int32_t maxContextLength)
Method for subclasses to use to set the maximum context length. More...

UChar filteredCharAt (const Replaceable& text, int32_t i) const
Method for subclasses to use to obtain a character in the given string, with filtering.


Friends

class  CompoundTransliterator

Detailed Description

Transliterator is an abstract class that transliterates text from one format to another.

The most common kind of transliterator is a script, or alphabet, transliterator. For example, a Russian to Latin transliterator changes Russian text written in Cyrillic characters to phonetically equivalent Latin characters. It does not translate Russian to English! Transliteration, unlike translation, operates on characters, without reference to the meanings of words and sentences.

Although script conversion is its most common use, a transliterator can actually perform a more general class of tasks. In fact, Transliterator defines a very general API which specifies only that a segment of the input text is replaced by new text. The particulars of this conversion are determined entirely by subclasses of Transliterator.

Transliterators are stateless

Transliterator objects are stateless; they retain no information between calls to transliterate(). (However, this does not mean that threads may share transliterators without synchronizing them. Transliterators are not immutable, so they must be synchronized when shared between threads.) This1 might seem to limit the complexity of the transliteration operation. In practice, subclasses perform complex transliterations by delaying the replacement of text until it is known that no other replacements are possible. In other words, although the Transliterator objects are stateless, the source text itself embodies all the needed information, and delayed operation allows arbitrary complexity.

Batch transliteration

The simplest way to perform transliteration is all at once, on a string of existing text. This is referred to as batch transliteration. For example, given a string input and a transliterator t, the call

String result = t.transliterate(input);

will transliterate it and return the result. Other methods allow the client to specify a substring to be transliterated and to use Replaceable objects instead of strings, in order to preserve out-of-band information (such as text styles).

Keyboard transliteration

Somewhat more involved is keyboard, or incremental transliteration. This is the transliteration of text that is arriving from some source (typically the user's keyboard) one character at a time, or in some other piecemeal fashion.

In keyboard transliteration, a Replaceable buffer stores the text. As text is inserted, as much as possible is transliterated on the fly. This means a GUI that displays the contents of the buffer may show text being modified as each new character arrives.

Consider the simple RuleBasedTransliterator:

th>{theta}
t>{tau}

When the user types 't', nothing will happen, since the transliterator is waiting to see if the next character is 'h'. To remedy this, we introduce the notion of a cursor, marked by a '|' in the output string:

t>|{tau}
{tau}h>{theta}

Now when the user types 't', tau appears, and if the next character is 'h', the tau changes to a theta. This is accomplished by maintaining a cursor position (independent of the insertion point, and invisible in the GUI) across calls to transliterate(). Typically, the cursor will be coincident with the insertion point, but in a case like the one above, it will precede the insertion point.

Keyboard transliteration methods maintain a set of three indices that are updated with each call to transliterate(), including the cursor, start, and limit. Since these indices are changed by the method, they are passed in an int[] array. The START index marks the beginning of the substring that the transliterator will look at. It is advanced as text becomes committed (but it is not the committed index; that's the CURSOR). The CURSOR index, described above, marks the point at which the transliterator last stopped, either because it reached the end, or because it required more characters to disambiguate between possible inputs. The CURSOR can also be explicitly set by rules in a RuleBasedTransliterator. Any characters before the CURSOR index are frozen; future keyboard transliteration calls within this input sequence will not change them. New text is inserted at the LIMIT index, which marks the end of the substring that the transliterator looks at.

Because keyboard transliteration assumes that more characters are to arrive, it is conservative in its operation. It only transliterates when it can do so unambiguously. Otherwise it waits for more characters to arrive. When the client code knows that no more characters are forthcoming, perhaps because the user has performed some input termination operation, then it should call finishTransliteration() to complete any pending transliterations.

Inverses

Pairs of transliterators may be inverses of one another. For example, if transliterator A transliterates characters by incrementing their Unicode value (so "abc" -> "def"), and transliterator B decrements character values, then A is an inverse of B and vice versa. If we compose A with B in a compound transliterator, the result is the indentity transliterator, that is, a transliterator that does not change its input text.

The Transliterator method getInverse() returns a transliterator's inverse, if one exists, or null otherwise. However, the result of getInverse() usually will not be a true mathematical inverse. This is because true inverse transliterators are difficult to formulate. For example, consider two transliterators: AB, which transliterates the character 'A' to 'B', and BA, which transliterates 'B' to 'A'. It might seem that these are exact inverses, since

"A" x AB -> "B"
"B" x BA -> "A"

where 'x' represents transliteration. However,

"ABCD" x AB -> "BBCD"
"BBCD" x BA -> "AACD"

so AB composed with BA is not the identity. Nonetheless, BA may be usefully considered to be AB's inverse, and it is on this basis that AB.getInverse() could legitimately return BA.

IDs and display names

A transliterator is designated by a short identifier string or ID. IDs follow the format source-destination, where source describes the entity being replaced, and destination describes the entity replacing source. The entities may be the names of scripts, particular sequences of characters, or whatever else it is that the transliterator converts to or from. For example, a transliterator from Russian to Latin might be named "Russian-Latin". A transliterator from keyboard escape sequences to Latin-1 characters might be named "KeyboardEscape-Latin1". By convention, system entity names are in English, with the initial letters of words capitalized; user entity names may follow any format so long as they do not contain dashes.

In addition to programmatic IDs, transliterator objects have display names for presentation in user interfaces, returned by #getDisplayName.

Factory methods and registration

In general, client code should use the factory method getInstance() to obtain an instance of a transliterator given its ID. Valid IDs may be enumerated using getAvailableIDs(). Since transliterators are mutable, multiple calls to getInstance() with the same ID will return distinct objects.

In addition to the system transliterators registered at startup, user transliterators may be registered by calling registerInstance() at run time. A registered instance acts a template; future calls to getInstance() with the ID of the registered object return clones of that object. Thus any object passed to registerInstance() must implement clone() propertly. To register a transliterator subclass without instantiating it (until it is needed), users may call registerClass(). In this case, the objects are instantiated by invoking the zero-argument public constructor of the class.

Subclassing

Subclasses must implement the abstract method handleTransliterate().

Subclasses should override the transliterate() method taking a Replaceable and the transliterate() method taking a String and StringBuffer if the performance of these methods can be improved over the performance obtained by the default implementations in this class.

Author(s):
Alan Liu
Draft:

Member Enumeration Documentation

enum Transliterator::Direction

Direction constant indicating the direction in a transliterator, e.g., the forward or reverse rules of a RuleBasedTransliterator.

An "A-B" transliterator transliterates A to B when operating in the forward direction, and B to A when operating in the reverse direction.

Draft:
Enumeration values:
FORWARD  
REVERSE  

Member Function Documentation

virtual Transliterator::~Transliterator () [virtual]

Destructor.

Draft:

virtual Transliterator * Transliterator::clone (void) const [inline, virtual]

Implements Cloneable.

All subclasses are encouraged to implement this method if it is possible and reasonable to do so. Subclasses that are to be registered with the system using registerInstance() are required to implement this method. If a subclass does not implement clone() properly and is registered with the system using registerInstance(), then the default clone() implementation will return null, and calls to createInstance() will fail.

See also:
registerInstance()
Draft:

Reimplemented in CompoundTransliterator, HangulJamoTransliterator, HexToUnicodeTransliterator, JamoHangulTransliterator, NullTransliterator, RuleBasedTransliterator, and UnicodeToHexTransliterator.

virtual int32_t Transliterator::transliterate (Replaceable & text, int32_t start, int32_t limit) const [virtual]

Transliterates a segment of a string, with optional filtering.

Parameters:
text   the string to be transliterated
start   the beginning index, inclusive; 0 <= start <= limit.
limit   the ending index, exclusive; start <= limit <= text.length().
filter   the filter. Any character for which filter.contains() returns false will not be altered by this transliterator. If filter is null then no filtering is applied.
Returns:
The new limit index. The text previously occupying [start, limit) has been transliterated, possibly to a string of a different length, at [start, new-limit), where new-limit is the return value.
Draft:

Reimplemented in NullTransliterator.

virtual void Transliterator::transliterate (Replaceable & text) const [virtual]

Transliterates an entire string in place.

Convenience method.

Parameters:
text   the string to be transliterated
Draft:

virtual void Transliterator::transliterate (Replaceable & text, Position & index, const UnicodeString & insertion, UErrorCode & status) const [virtual]

Transliterates the portion of the text buffer that can be transliterated unambiguosly after new text has been inserted, typically as a result of a keyboard event.

The new text in insertion will be inserted into text at index.limit, advancing index.limit by insertion.length(). Then the transliterator will try to transliterate characters of text between index.cursor and index.limit. Characters before index.cursor will not be changed.

Upon return, values in index will be updated. index.start will be advanced to the first character that future calls to this method will read. index.cursor and index.limit will be adjusted to delimit the range of text that future calls to this method may change.

Typical usage of this method begins with an initial call with index.start and index.limit set to indicate the portion of text to be transliterated, and index.cursor == index.start. Thereafter, index can be used without modification in future calls, provided that all changes to text are made via this method.

This method assumes that future calls may be made that will insert new text into the buffer. As a result, it only performs unambiguous transliterations. After the last call to this method, there may be untransliterated text that is waiting for more input to resolve an ambiguity. In order to perform these pending transliterations, clients should call #finishTransliteration after the last call to this method has been made.

Parameters:
text   the buffer holding transliterated and untransliterated text
index   an array of three integers.

  • index.start: the beginning index, inclusive; 0 <= index.start <= index.limit.

  • index.limit: the ending index, exclusive; index.start <= index.limit <= text.length(). insertion is inserted at index.limit.

  • index.cursor: the next character to be considered for transliteration; index.start <= index.cursor <= index.limit. Characters before index.cursor will not be changed by future calls to this method.

Parameters:
insertion   text to be inserted and possibly transliterated into the translation buffer at index.limit. If null then no text is inserted.
See also:
handleTransliterate()
Exceptions:
IllegalArgumentException   if index is invalid
Draft:

virtual void Transliterator::transliterate (Replaceable & text, Position & index, UChar insertion, UErrorCode & status) const [virtual]

Transliterates the portion of the text buffer that can be transliterated unambiguosly after a new character has been inserted, typically as a result of a keyboard event.

This is a convenience method; see #transliterate(Replaceable for details.

Parameters:
text   the buffer holding transliterated and untransliterated text
index   an array of three integers. See #transliterate(Replaceable.
insertion   text to be inserted and possibly transliterated into the translation buffer at index.limit.
See also:
transliterate()(Replaceable, int[], String)
Draft:

virtual void Transliterator::transliterate (Replaceable & text, Position & index, UErrorCode & status) const [virtual]

Transliterates the portion of the text buffer that can be transliterated unambiguosly.

This is a convenience method; see #transliterate(Replaceable for details.

Parameters:
text   the buffer holding transliterated and untransliterated text
index   an array of three integers. See #transliterate(Replaceable.
See also:
transliterate()(Replaceable, int[], String)
Draft:

virtual void Transliterator::finishTransliteration (Replaceable & text, Position & index) const [virtual]

Finishes any pending transliterations that were waiting for more characters.

Clients should call this method as the last call after a sequence of one or more calls to transliterate().

Parameters:
text   the buffer holding transliterated and untransliterated text.
index   the array of indices previously passed to #transliterate
Draft:

int32_t Transliterator::getMaximumContextLength (void) const [inline]

Returns the length of the longest context required by this transliterator.

This is preceding context. The default implementation supplied by Transliterator returns zero; subclasses that use preceding context should override this method to return the correct value. For example, if a transliterator translates "ddd" (where d is any digit) to "555" when preceded by "(ddd)", then the preceding context length is 5, the length of "(ddd)".

Returns:
The maximum number of preceding context characters this transliterator needs to examine
Draft:

virtual const UnicodeString & Transliterator::getID (void) const [virtual]

Returns a programmatic identifier for this transliterator.

If this identifier is passed to getInstance(), it will return this object, if it has been registered.

See also:
registerInstance() , registerClass , getAvailableIDs
Draft:

virtual const UnicodeFilter * Transliterator::getFilter (void) const [virtual]

Returns the filter used by this transliterator, or null if this transliterator uses no filter.

Draft:

virtual void Transliterator::adoptFilter (UnicodeFilter * adoptedFilter) [virtual]

Changes the filter used by this transliterator.

If the filter is set to null then no filtering will occur.

Callers must take care if a transliterator is in use by multiple threads. The filter should not be changed by one thread while another thread may be transliterating.

Draft:

Transliterator * Transliterator::createInverse (void) const

Returns this transliterator's inverse.

See the class documentation for details. This implementation simply inverts the two entities in the ID and attempts to retrieve the resulting transliterator. That is, if getID() returns "A-B", then this method will return the result of getInstance("B-A"), or null if that call fails.

This method does not take filtering into account. The returned transliterator will have no filter.

Subclasses with knowledge of their inverse may wish to override this method.

Returns:
a transliterator that is an inverse, not necessarily exact, of this transliterator, or null if no such transliterator is registered.
See also:
registerInstance()
Draft:

UnicodeString & Transliterator::getDisplayName (const UnicodeString & ID, UnicodeString & result) [static]

Returns a name for this transliterator that is appropriate for display to the user in the default locale.

See #getDisplayName(Locale) for details.

Draft:

UnicodeString & Transliterator::getDisplayName (const UnicodeString & ID, const Locale & inLocale, UnicodeString & result) [static]

Returns a name for this transliterator that is appropriate for display to the user in the given locale.

This name is taken from the locale resource data in the standard manner of the java.text package.

If no localized names exist in the system resource bundles, a name is synthesized using a localized MessageFormat pattern from the resource data. The arguments to this pattern are an integer followed by one or two strings. The integer is the number of strings, either 1 or 2. The strings are formed by splitting the ID for this transliterator at the first '-'. If there is no '-', then the entire ID forms the only string.

Parameters:
inLocale   the Locale in which the display name should be localized.
See also:
java.text.MessageFormat
Draft:

Transliterator * Transliterator::createInstance (const UnicodeString & ID, Direction dir = FORWARD) [static]

Returns a Transliterator object given its ID.

The ID must be either a system transliterator ID or a ID registered using registerInstance().

Parameters:
ID   a valid ID, as enumerated by getAvailableIDs()
Returns:
A Transliterator object with the given ID
Exceptions:
IllegalArgumentException   if the given ID is invalid.
See also:
registerInstance() , getAvailableIDs , getID()
Draft:

void Transliterator::registerInstance (Transliterator * adoptedObj, UErrorCode & status) [static]

Registers a instance obj of a subclass of Transliterator with the system.

When createInstance() is called with an ID string that is equal to obj->getID(), then obj->clone() is returned.

After this call the Transliterator class owns the adoptedObj and will delete it.

Parameters:
obj   an instance of subclass of Transliterator that defines clone()
See also:
getInstance , registerClass , unregister()
Draft:

void Transliterator::unregister (const UnicodeString & ID) [static]

Unregisters a transliterator or class.

This may be either a system transliterator or a user transliterator or class.

Parameters:
ID   the ID of the transliterator or class
Returns:
the Object that was registered with ID, or null if none was
See also:
registerInstance() , registerClass
Draft:

int32_t Transliterator::countAvailableIDs (void) [static]

Return the number of IDs currently registered with the system.

To retrieve the actual IDs, call getAvailableID(i) with i from 0 to countAvailableIDs() - 1.

Draft:

const UnicodeString & Transliterator::getAvailableID (int32_t index) [static]

Return the index-th available ID.

index must be between 0 and countAvailableIDs() - 1, inclusive. If index is out of range, the result of getAvailableID(0) is returned.

Draft:

Transliterator::Transliterator (const UnicodeString & ID, UnicodeFilter * adoptedFilter) [protected]

Default constructor.

Parameters:
ID   the string identifier for this transliterator
adoptedFilter   the filter. Any character for which filter.contains() returns false will not be altered by this transliterator. If filter is null then no filtering is applied.

Transliterator::Transliterator (const Transliterator &) [protected]

Copy constructor.

Transliterator & Transliterator::operator= (const Transliterator &) [protected]

Assignment operator.

virtual void Transliterator::handleTransliterate (Replaceable & text, Position & offsets, bool_t isIncremental) const [protected, pure virtual]

Abstract method that concrete subclasses define to implement keyboard transliteration.

This method should transliterate all characters between index.cursor and index.limit that can be unambiguously transliterated, regardless of future insertions of text at index.limit. index.cursor should be advanced past committed characters (those that will not change in future calls to this method). index.limit should be updated to reflect text replacements that shorten or lengthen the text between index.cursor and index.limit. Upon return, neither index.cursor nor index.limit should be less than the initial value of index.cursor. index.start should not be changed.

Parameters:
text   the buffer holding transliterated and untransliterated text
index   an array of three integers. See #transliterate(Replaceable.
See also:
transliterate()

Reimplemented in CompoundTransliterator, HangulJamoTransliterator, HexToUnicodeTransliterator, JamoHangulTransliterator, NullTransliterator, RuleBasedTransliterator, and UnicodeToHexTransliterator.

void Transliterator::setMaximumContextLength (int32_t maxContextLength) [protected]

Method for subclasses to use to set the maximum context length.

See also:
getMaximumContextLength()

UChar Transliterator::filteredCharAt (const Replaceable & text, int32_t i) const [protected]

Method for subclasses to use to obtain a character in the given string, with filtering.


Friends And Related Function Documentation

friend class CompoundTransliterator [friend]


The documentation for this class was generated from the following file:
Generated at Thu Feb 10 15:31:04 2000 for icu by doxygen 1.0.0 written by Dimitri van Heesch, © 1997-1999