com.ibm.text
Class UnicodeSet

java.lang.Object
  |
  +--com.ibm.text.UnicodeFilter
        |
        +--com.ibm.text.UnicodeSet
All Implemented Interfaces:
UnicodeMatcher

public class UnicodeSet
extends UnicodeFilter

A mutable set of Unicode characters. Objects of this class represent character classes used in regular expressions. A character specifies a subset of Unicode code points. Legal code points are U+0000 to U+10FFFF, inclusive.

UnicodeSet supports two APIs. The first is the operand API that allows the caller to modify the value of a UnicodeSet object. It conforms to Java 2's java.util.Set interface, although UnicodeSet does not actually implement that interface. All methods of Set are supported, with the modification that they take a character range or single character instead of an Object, and they take a UnicodeSet instead of a Collection. The operand API may be thought of in terms of boolean logic: a boolean OR is implemented by add, a boolean AND is implemented by retain, a boolean XOR is implemented by complement taking an argument, and a boolean NOT is implemented by complement with no argument. In terms of traditional set theory function names, add is a union, retain is an intersection, remove is an asymmetric difference, and complement with no argument is a set complement with respect to the superset range MIN_VALUE-MAX_VALUE

The second API is the applyPattern()/toPattern() API from the java.text.Format-derived classes. Unlike the methods that add characters, add categories, and control the logic of the set, the method applyPattern() sets all attributes of a UnicodeSet at once, based on a string pattern.

Pattern syntax

Patterns are accepted by the constructors and the applyPattern() methods and returned by the toPattern() method. These patterns follow a syntax similar to that employed by version 8 regular expression character classes:
pattern :=  ('[' '^'? item* ']') | property
item :=  char | (char '-' char) | pattern-expr
pattern-expr :=  pattern | pattern-expr pattern | pattern-expr op pattern
op :=  '&' | '-'
special :=  '[' | ']' | '-'
char :=  any character that is not special
| ('\'
any character)
| ('\u' hex hex hex hex)
hex :=  any character for which Character.digit(c, 16) returns a non-negative result
property :=  a Unicode property set pattern

Legend:
a := b   a may be replaced by b
a? zero or one instance of a
a* one or more instances of a
a | b either a or b
'a' the literal string between the quotes
Any character may be preceded by a backslash in order to remove any special meaning. White space characters, as defined by UCharacter.isWhitespace(), are ignored, unless they are escaped.

Property patterns specify a set of characters having a certain property as defined by the Unicode standard. Both the POSIX-like "[:Lu:]" and the Perl-like syntax "\p{Lu}" are recognized. For a complete list of supported property patterns, see the User's Guide for UnicodeSet at http://oss.software.ibm.com/icu/userguide/unicodeset.html. Actual determination of property data is defined by the underlying Unicode database as implemented by UCharacter.

Patterns specify individual characters, ranges of characters, and Unicode property sets. When elements are concatenated, they specify their union. To complement a set, place a '^' immediately after the opening '['. Property patterns are inverted by modifying their delimiters; "[:^foo]" and "\P{foo}". In any other location, '^' has no special meaning.

Ranges are indicated by placing two a '-' between two characters, as in "a-z". This specifies the range of all characters from the left to the right, in Unicode order. If the left character is greater than or equal to the right character it is a syntax error. If a '-' occurs as the first character after the opening '[' or '[^', or if it occurs as the last character before the closing ']', then it is taken as a literal. Thus "[a\-b]", "[-ab]", and "[ab-]" all indicate the same set of three characters, 'a', 'b', and '-'.

Sets may be intersected using the '&' operator or the asymmetric set difference may be taken using the '-' operator, for example, "[[:L:]&[\u0000-\u0FFF]]" indicates the set of all Unicode letters with values less than 4096. Operators ('&' and '|') have equal precedence and bind left-to-right. Thus "[[:L:]-[a-z]-[\u0100-\u01FF]]" is equivalent to "[[[:L:]-[a-z]]-[\u0100-\u01FF]]". This only really matters for difference; intersection is commutative.
[a]The set containing 'a'
[a-z]The set containing 'a' through 'z' and all letters in between, in Unicode order
[^a-z]The set containing all characters but 'a' through 'z', that is, U+0000 through 'a'-1 and 'z'+1 through U+10FFFF
[[pat1][pat2]] The union of sets specified by pat1 and pat2
[[pat1]&[pat2]] The intersection of sets specified by pat1 and pat2
[[pat1]-[pat2]] The asymmetric difference of sets specified by pat1 and pat2
[:Lu:] or \p{Lu} The set of characters having the specified Unicode property; in this case, Unicode uppercase letters
[:^Lu:] or \P{Lu} The set of characters not having the given Unicode property

Version:
$RCSfile: UnicodeSet.java,v $ $Revision: 1.53 $ $Date: 2001/12/03 21:33:59 $
Author:
Alan Liu

Field Summary
static int MAX_VALUE
          Maximum value that can be stored in a UnicodeSet.
static int MIN_VALUE
          Minimum value that can be stored in a UnicodeSet.
 
Fields inherited from interface com.ibm.text.UnicodeMatcher
U_MATCH, U_MISMATCH, U_PARTIAL_MATCH
 
Constructor Summary
UnicodeSet()
          Constructs an empty set.
UnicodeSet(int category)
          Deprecated. this will be removed Dec-31-2002
UnicodeSet(int start, int end)
          Constructs a set containing the given range.
UnicodeSet(java.lang.String pattern)
          Constructs a set from the given pattern.
UnicodeSet(java.lang.String pattern, boolean ignoreWhitespace)
          Constructs a set from the given pattern.
UnicodeSet(java.lang.String pattern, java.text.ParsePosition pos, SymbolTable symbols)
          Constructs a set from the given pattern.
UnicodeSet(UnicodeSet other)
          Constructs a copy of an existing set.
 
Method Summary
 java.lang.StringBuffer _generatePattern(java.lang.StringBuffer result, boolean escapeUnprintable)
          Generate and append a string representation of this set to result.
 void add(int c)
          Adds the specified character to this set if it is not already present.
 void add(int start, int end)
          Adds the specified range to this set if it is not already present.
 void addAll(UnicodeSet c)
          Adds all of the elements in the specified set to this set if they're not already present.
 void applyPattern(java.lang.String pattern)
          Modifies this set to represent the set specified by the given pattern.
 void applyPattern(java.lang.String pattern, boolean ignoreWhitespace)
          Modifies this set to represent the set specified by the given pattern, optionally ignoring whitespace.
 int charAt(int index)
          Returns the character at the given index within this set, where the set is ordered by ascending code point.
 void clear()
          Removes all of the elements from this set.
 java.lang.Object clone()
          Return a new set that is equivalent to this one.
 void compact()
          Reallocate this objects internal structures to take up the least possible space, without changing this object's value.
 void complement()
          Inverts this set.
 void complement(int c)
          Complements the specified character in this set.
 void complement(int start, int end)
          Complements the specified range in this set.
 void complementAll(UnicodeSet c)
          Complements in this set all elements contained in the specified set.
 boolean contains(int c)
          Returns true if this set contains the specified char.
 boolean contains(int start, int end)
          Returns true if this set contains every character in the specified range of chars.
 boolean containsAll(UnicodeSet c)
          Returns true if the specified set is a subset of this set.
 boolean equals(java.lang.Object o)
          Compares the specified object with this set for equality.
 UnicodeSet getMatchSet(UnicodeSet toUnionTo)
          Implementation of UnicodeMatcher API.
 int getRangeCount()
          Iteration method that returns the number of ranges contained in this set.
 int getRangeEnd(int index)
          Iteration method that returns the last character in the specified range of this set.
 int getRangeStart(int index)
          Iteration method that returns the first character in the specified range of this set.
 int hashCode()
          Returns the hash code value for this set.
 int indexOf(int c)
          Returns the index of the given character within this set, where the set is ordered by ascending code point.
 boolean isEmpty()
          Returns true if this set contains no elements.
 int matches(Replaceable text, int[] offset, int limit, boolean incremental)
          Implementation of UnicodeMatcher.matches().
 boolean matchesIndexValue(int v)
          Implementation of UnicodeMatcher API.
 void remove(int c)
          Removes the specified character from this set if it is present.
 void remove(int start, int end)
          Removes the specified range from this set if it is present.
 void removeAll(UnicodeSet c)
          Removes from this set all of its elements that are contained in the specified set.
static boolean resemblesPattern(java.lang.String pattern, int pos)
          Return true if the given position, in the given pattern, appears to be the start of a UnicodeSet pattern.
 void retain(int c)
          Retain the specified character from this set if it is present.
 void retain(int start, int end)
          Retain only the elements in this set that are contained in the specified range.
 void retainAll(UnicodeSet c)
          Retains only the elements in this set that are contained in the specified set.
 void set(int start, int end)
          Make this object represent the range start - end.
 void set(UnicodeSet other)
          Make this object represent the same set as other.
 int size()
          Returns the number of elements in this set (its cardinality), n, where 0 <= n <= 65536.
 java.lang.String toPattern(boolean escapeUnprintable)
          Returns a string representation of this set.
 java.lang.String toString()
          Return a programmer-readable string representation of this object.
 
Methods inherited from class java.lang.Object
finalize, getClass, notify, notifyAll, wait, wait, wait
 

Field Detail

MIN_VALUE

public static final int MIN_VALUE
Minimum value that can be stored in a UnicodeSet.

MAX_VALUE

public static final int MAX_VALUE
Maximum value that can be stored in a UnicodeSet.
Constructor Detail

UnicodeSet

public UnicodeSet()
Constructs an empty set.

UnicodeSet

public UnicodeSet(UnicodeSet other)
Constructs a copy of an existing set.

UnicodeSet

public UnicodeSet(int start,
                  int end)
Constructs a set containing the given range. If end > start then an empty set is created.
Parameters:
start - first character, inclusive, of range
end - last character, inclusive, of range

UnicodeSet

public UnicodeSet(java.lang.String pattern)
Constructs a set from the given pattern. See the class description for the syntax of the pattern language. Whitespace is ignored.
Parameters:
pattern - a string specifying what characters are in the set
Throws:
java.lang.IllegalArgumentException - if the pattern contains a syntax error.

UnicodeSet

public UnicodeSet(java.lang.String pattern,
                  boolean ignoreWhitespace)
Constructs a set from the given pattern. See the class description for the syntax of the pattern language.
Parameters:
pattern - a string specifying what characters are in the set
ignoreWhitespace - if true, ignore characters for which Character.isWhitespace() returns true
Throws:
java.lang.IllegalArgumentException - if the pattern contains a syntax error.

UnicodeSet

public UnicodeSet(java.lang.String pattern,
                  java.text.ParsePosition pos,
                  SymbolTable symbols)
Constructs a set from the given pattern. See the class description for the syntax of the pattern language.
Parameters:
pattern - a string specifying what characters are in the set
pos - on input, the position in pattern at which to start parsing. On output, the position after the last character parsed.
symbols - a symbol table mapping variables to char[] arrays and chars to UnicodeSets
Throws:
java.lang.IllegalArgumentException - if the pattern contains a syntax error.

UnicodeSet

public UnicodeSet(int category)
Deprecated. this will be removed Dec-31-2002

DEPRECATED - Constructs a set from the given Unicode character category.
Parameters:
category - an integer indicating the character category as returned by java.lang.Character.getType(). Note that this is different from the UCharacterCategory codes.
Throws:
java.lang.IllegalArgumentException - if the given category is invalid.
Method Detail

clone

public java.lang.Object clone()
Return a new set that is equivalent to this one.
Overrides:
clone in class java.lang.Object

set

public void set(int start,
                int end)
Make this object represent the range start - end. If end > start then this object is set to an an empty range.
Parameters:
start - first character in the set, inclusive

set

public void set(UnicodeSet other)
Make this object represent the same set as other.
Parameters:
other - a UnicodeSet whose value will be copied to this object

applyPattern

public final void applyPattern(java.lang.String pattern)
Modifies this set to represent the set specified by the given pattern. See the class description for the syntax of the pattern language. Whitespace is ignored.
Parameters:
pattern - a string specifying what characters are in the set
Throws:
java.lang.IllegalArgumentException - if the pattern contains a syntax error.

applyPattern

public void applyPattern(java.lang.String pattern,
                         boolean ignoreWhitespace)
Modifies this set to represent the set specified by the given pattern, optionally ignoring whitespace. See the class description for the syntax of the pattern language.
Parameters:
pattern - a string specifying what characters are in the set
ignoreWhitespace - if true then characters for which Character.isWhitespace() returns true are ignored
Throws:
java.lang.IllegalArgumentException - if the pattern contains a syntax error.

resemblesPattern

public static boolean resemblesPattern(java.lang.String pattern,
                                       int pos)
Return true if the given position, in the given pattern, appears to be the start of a UnicodeSet pattern.

toPattern

public java.lang.String toPattern(boolean escapeUnprintable)
Returns a string representation of this set. If the result of calling this function is passed to a UnicodeSet constructor, it will produce another set that is equal to this one.
Overrides:
toPattern in class UnicodeFilter
Following copied from interface: com.ibm.text.UnicodeMatcher
Parameters:
escapeUnprintable - if TRUE then convert unprintable character to their hex escape representations, \\uxxxx or \\Uxxxxxxxx. Unprintable characters are those other than U+000A, U+0020..U+007E.

_generatePattern

public java.lang.StringBuffer _generatePattern(java.lang.StringBuffer result,
                                               boolean escapeUnprintable)
Generate and append a string representation of this set to result. This does not use this.pat, the cleaned up copy of the string passed to applyPattern().

size

public int size()
Returns the number of elements in this set (its cardinality), n, where 0 <= n <= 65536.
Returns:
the number of elements in this set (its cardinality).

isEmpty

public boolean isEmpty()
Returns true if this set contains no elements.
Returns:
true if this set contains no elements.

contains

public boolean contains(int start,
                        int end)
Returns true if this set contains every character in the specified range of chars. If end > start then the results of this method are undefined.
Returns:
true if this set contains the specified range of chars.

matchesIndexValue

public boolean matchesIndexValue(int v)
Implementation of UnicodeMatcher API. Returns true if this set contains any character whose low byte is the given value. This is used by RuleBasedTransliterator for indexing.
Overrides:
matchesIndexValue in class UnicodeFilter

matches

public int matches(Replaceable text,
                   int[] offset,
                   int limit,
                   boolean incremental)
Implementation of UnicodeMatcher.matches().
Overrides:
matches in class UnicodeFilter
Following copied from interface: com.ibm.text.UnicodeMatcher
Parameters:
text - the text to be matched
offset - on input, the index into text at which to begin matching. On output, the limit of the matched text. The number of matched characters is the output value of offset minus the input value. Offset should always point to the HIGH SURROGATE (leading code unit) of a pair of surrogates, both on entry and upon return.
limit - the limit index of text to be matched. Greater than offset for a forward direction match, less than offset for a backward direction match. The last character to be considered for matching will be text.charAt(limit-1) in the forward direction or text.charAt(limit+1) in the backward direction.
incremental - if TRUE, then assume further characters may be inserted at limit and check for partial matching. Otherwise assume the text as given is complete.
Returns:
a match degree value indicating a full match, a partial match, or a mismatch. If incremental is FALSE then U_PARTIAL_MATCH should never be returned.

getMatchSet

public UnicodeSet getMatchSet(UnicodeSet toUnionTo)
Implementation of UnicodeMatcher API. Union the set of all characters that may be matched by this object into the given set.
Overrides:
getMatchSet in class UnicodeFilter
Parameters:
toUnionTo - the set into which to union the source characters
Returns:
a reference to toUnionTo

indexOf

public int indexOf(int c)
Returns the index of the given character within this set, where the set is ordered by ascending code point. If the character is not in this set, return -1. The inverse of this method is charAt().
Returns:
an index from 0..size()-1, or -1

charAt

public int charAt(int index)
Returns the character at the given index within this set, where the set is ordered by ascending code point. If the index is out of range, return -1. The inverse of this method is indexOf().
Parameters:
index - an index from 0..size()-1
Returns:
the character at the given index, or -1.

contains

public boolean contains(int c)
Returns true if this set contains the specified char.
Overrides:
contains in class UnicodeFilter
Returns:
true if this set contains the specified char.

add

public void add(int start,
                int end)
Adds the specified range to this set if it is not already present. If this set already contains the specified range, the call leaves this set unchanged. If end > start then an empty range is added, leaving the set unchanged.
Parameters:
start - first character, inclusive, of range to be added to this set.
end - last character, inclusive, of range to be added to this set.

add

public final void add(int c)
Adds the specified character to this set if it is not already present. If this set already contains the specified character, the call leaves this set unchanged.

retain

public void retain(int start,
                   int end)
Retain only the elements in this set that are contained in the specified range. If end > start then an empty range is retained, leaving the set empty.
Parameters:
start - first character, inclusive, of range to be retained to this set.
end - last character, inclusive, of range to be retained to this set.

retain

public final void retain(int c)
Retain the specified character from this set if it is present.

remove

public void remove(int start,
                   int end)
Removes the specified range from this set if it is present. The set will not contain the specified range once the call returns. If end > start then an empty range is removed, leaving the set unchanged.
Parameters:
start - first character, inclusive, of range to be removed from this set.
end - last character, inclusive, of range to be removed from this set.

remove

public final void remove(int c)
Removes the specified character from this set if it is present. The set will not contain the specified character once the call returns.

complement

public void complement(int start,
                       int end)
Complements the specified range in this set. Any character in the range will be removed if it is in this set, or will be added if it is not in this set. If end > start then an empty range is complemented, leaving the set unchanged.
Parameters:
start - first character, inclusive, of range to be removed from this set.
end - last character, inclusive, of range to be removed from this set.

complement

public final void complement(int c)
Complements the specified character in this set. The character will be removed if it is in this set, or will be added if it is not in this set.

complement

public void complement()
Inverts this set. This operation modifies this set so that its value is its complement. This is equivalent to complement(MIN_VALUE, MAX_VALUE).

containsAll

public boolean containsAll(UnicodeSet c)
Returns true if the specified set is a subset of this set.
Parameters:
c - set to be checked for containment in this set.
Returns:
true if this set contains all of the elements of the specified set.

addAll

public void addAll(UnicodeSet c)
Adds all of the elements in the specified set to this set if they're not already present. This operation effectively modifies this set so that its value is the union of the two sets. The behavior of this operation is unspecified if the specified collection is modified while the operation is in progress.
Parameters:
c - set whose elements are to be added to this set.

retainAll

public void retainAll(UnicodeSet c)
Retains only the elements in this set that are contained in the specified set. In other words, removes from this set all of its elements that are not contained in the specified set. This operation effectively modifies this set so that its value is the intersection of the two sets.
Parameters:
c - set that defines which elements this set will retain.

removeAll

public void removeAll(UnicodeSet c)
Removes from this set all of its elements that are contained in the specified set. This operation effectively modifies this set so that its value is the asymmetric set difference of the two sets.
Parameters:
c - set that defines which elements will be removed from this set.

complementAll

public void complementAll(UnicodeSet c)
Complements in this set all elements contained in the specified set. Any character in the other set will be removed if it is in this set, or will be added if it is not in this set.
Parameters:
c - set that defines which elements will be complemented from this set.

clear

public void clear()
Removes all of the elements from this set. This set will be empty after this call returns.

getRangeCount

public int getRangeCount()
Iteration method that returns the number of ranges contained in this set.
See Also:
getRangeStart(int), getRangeEnd(int)

getRangeStart

public int getRangeStart(int index)
Iteration method that returns the first character in the specified range of this set.
Throws:
ArrayIndexOutOfBoundsException - if index is outside the range 0..getRangeCount()-1
See Also:
getRangeCount(), getRangeEnd(int)

getRangeEnd

public int getRangeEnd(int index)
Iteration method that returns the last character in the specified range of this set.
Throws:
ArrayIndexOutOfBoundsException - if index is outside the range 0..getRangeCount()-1
See Also:
getRangeStart(int), getRangeEnd(int)

compact

public void compact()
Reallocate this objects internal structures to take up the least possible space, without changing this object's value.

equals

public boolean equals(java.lang.Object o)
Compares the specified object with this set for equality. Returns true if the specified object is also a set, the two sets have the same size, and every member of the specified set is contained in this set (or equivalently, every member of this set is contained in the specified set).
Overrides:
equals in class java.lang.Object
Parameters:
o - Object to be compared for equality with this set.
Returns:
true if the specified Object is equal to this set.

hashCode

public int hashCode()
Returns the hash code value for this set.
Overrides:
hashCode in class java.lang.Object
Returns:
the hash code value for this set.
See Also:
Object.hashCode()

toString

public java.lang.String toString()
Return a programmer-readable string representation of this object.
Overrides:
toString in class java.lang.Object


Copyright (c) 2001 IBM Corporation and others.