|
A regular expression is a formula for matching strings that
follow some pattern. Many people are afraid to use them because
they can look confusing and complicated. However, with a little
practice, it's pretty easy to write the expressions to make the
advanced web filters in Check&Get.
We are prepared some easy
examples that could help you to understand regular expression
use and start making your own web filters.
This document describes the formal Regular Expression
syntax, used in Check&Get
See also:
Table Of Context
- Regular Expression Syntax;
- Order of
Precedence
- Character Matching
- Bracket Expressions
- Quantifiers and Associated Meanings
-
Anchors
- Alternation and Grouping
Regular Expression Syntax
A regular expression is a pattern of text that consists of
ordinary characters (for example, letters a through z) and special
characters, known as metacharacters. The pattern describes
one or more strings to match when searching a body of text. The
regular expression serves as a template for matching a character
pattern to the string being searched.
Here are some examples of regular expression you might
encounter:
|
Regular Expression
|
Matches
|
|
^\s*$
|
Match a blank line.
|
|
\d{2}-\d{5}
|
Validate an ID number consisting f 2 digits, a hyphen, and
another 5 digits.
|
The following table contains the complete list of metacharacters
and their behavior in the context of regular expressions:
| Character
|
Description
|
|
\
|
Marks the next character as either a special character, a
literal, a backreference, or an octal escape. For example, 'n'
matches the character "n". '\n' matches a newline character. The
sequence '\\' matches "\" and "\(" matches "(".
|
|
^
|
Matches the position at the beginning of the input string. If
the RegExp object's Multiline
property is set, ^ also matches the position following '\n' or
'\r'.
|
|
$
|
Matches the position at the end of the input string. If the
RegExp object's Multiline
property is set, $ also matches the position preceding '\n' or
'\r'.
|
|
*
|
Matches the preceding character or subexpression zero or more
times. For example, zo* matches "z" and "zoo". * is equivalent to
{0,}.
|
|
+
|
Matches the preceding character or subexpression one or more
times. For example, 'zo+' matches "zo" and "zoo", but not "z". + is
equivalent to {1,}.
|
|
?
|
Matches the preceding character or subexpression zero or one
time. For example, "do(es)?" matches the "do" in "do" or "does". ?
is equivalent to {0,1}
|
|
{n}
|
n is a nonnegative integer. Matches exactly n
times. For example, 'o{2}' does not match the 'o' in "Bob," but
matches the two o's in "food".
|
|
{n,}
|
n is a nonnegative integer. Matches at least n
times. For example, 'o{2,}' does not match the "o" in "Bob" and
matches all the o's in "foooood". 'o{1,}' is equivalent to 'o+'.
'o{0,}' is equivalent to 'o*'.
|
|
{n,m}
|
m and n are nonnegative integers, where
n <= m. Matches at least n and at
most m times. For example, "o{1,3}" matches the first
three o's in "fooooood". 'o{0,1}' is equivalent to 'o?'. Note that
you cannot put a space between the comma and the numbers.
|
|
?
|
When this character immediately follows any of the other
quantifiers (*, +, ?, {n}, {n,},
{n,m}), the matching pattern is non-greedy. A
non-greedy pattern matches as little of the searched string as
possible, whereas the default greedy pattern matches as much of the
searched string as possible. For example, in the string "oooo",
'o+?' matches a single "o", while 'o+' matches all 'o's.
|
|
.
|
Matches any single character except "\n". To match any character
including the '\n', use a pattern such as '[\s\S]'.
|
|
(pattern)
|
Matches pattern and captures the match. The captured
match can be retrieved from the resulting Matches collection, using
the SubMatches collection in VBScript or the
$0…$9 properties in JScript. To
match parentheses characters ( ), use '\(' or '\)'.
|
|
(?:pattern)
|
Matches pattern but does not capture the match, that
is, it is a non-capturing match that is not stored for possible
later use. This is useful for combining parts of a pattern with the
"or" character (|). For example, 'industr(?:y|ies) is a more
economical expression than 'industry|industries'.
|
|
(?=pattern)
|
Positive lookahead matches the search string at any point where
a string matching pattern begins. This is a non-capturing
match, that is, the match is not captured for possible later use.
For example 'Windows (?=95|98|NT|2000)' matches "Windows" in
"Windows 2000" but not "Windows" in "Windows 3.1". Lookaheads do
not consume characters, that is, after a match occurs, the search
for the next match begins immediately following the last match, not
after the characters that comprised the lookahead.
|
|
(?!pattern)
|
Negative lookahead matches the search string at any point where
a string not matching pattern begins. This is a
non-capturing match, that is, the match is not captured for
possible later use. For example 'Windows (?!95|98|NT|2000)' matches
"Windows" in "Windows 3.1" but does not match "Windows" in "Windows
2000". Lookaheads do not consume characters, that is, after a match
occurs, the search for the next match begins immediately following
the last match, not after the characters that comprised the
lookahead.
|
|
x|y
|
Matches either x or y. For example, 'z|food'
matches "z" or "food". '(z|f)ood' matches "zood" or "food".
|
|
[xyz]
|
A character set. Matches any one of the enclosed characters. For
example, '[abc]' matches the 'a' in "plain".
|
|
[^xyz]
|
A negative character set. Matches any character not enclosed.
For example, '[^abc]' matches the 'p' in "plain".
|
|
[a-z]
|
A range of characters. Matches any character in the specified
range. For example, '[a-z]' matches any lowercase alphabetic
character in the range 'a' through 'z'.
|
|
[^a-z]
|
A negative range characters. Matches any character not in the
specified range. For example, '[^a-z]' matches any character not in
the range 'a' through 'z'.
|
|
\b
|
Matches a word boundary, that is, the position between a word
and a space. For example, 'er\b' matches the 'er' in "never" but
not the 'er' in "verb".
|
|
\B
|
Matches a nonword boundary. 'er\B' matches the 'er' in "verb"
but not the 'er' in "never".
|
|
\cx
|
Matches the control character indicated by x. For
example, \cM matches a Control-M or carriage return character. The
value of x must be in the range of A-Z or a-z. If not, c
is assumed to be a literal 'c' character.
|
|
\d
|
Matches a digit character. Equivalent to [0-9].
|
|
\D
|
Matches a nondigit character. Equivalent to [^0-9].
|
|
\f
|
Matches a form-feed character. Equivalent to \x0c and \cL.
|
|
\n
|
Matches a newline character. Equivalent to \x0a and \cJ.
|
|
\r
|
Matches a carriage return character. Equivalent to \x0d and
\cM.
|
|
\s
|
Matches any whitespace character including space, tab,
form-feed, etc. Equivalent to [ \f\n\r\t\v].
|
|
\S
|
Matches any non-white space character. Equivalent to
[^ \f\n\r\t\v].
|
|
\t
|
Matches a tab character. Equivalent to \x09 and \cI.
|
|
\v
|
Matches a vertical tab character. Equivalent to \x0b and
\cK.
|
|
\w
|
Matches any word character including underscore. Equivalent to
'[A-Za-z0-9_]'.
|
|
\W
|
Matches any nonword character. Equivalent to
'[^A-Za-z0-9_]'.
|
|
\xn
|
Matches n, where n is a hexadecimal escape
value. Hexadecimal escape values must be exactly two digits long.
For example, '\x41' matches "A". '\x041' is equivalent to '\x04'
& "1". Allows ASCII codes to be used in regular
expressions.
|
|
\num
|
Matches num, where num is a positive integer.
A reference back to captured matches. For example, '(.)\1' matches
two consecutive identical characters.
|
|
\n
|
Identifies either an octal escape value or a backreference. If
\n is preceded by at least n captured
subexpressions, n is a backreference. Otherwise,
n is an octal escape value if n is an octal digit
(0-7).
|
|
\nm
|
Identifies either an octal escape value or a backreference. If
\nm is preceded by at least nm captured
subexpressions, nm is a backreference. If \nm is
preceded by at least n captures, n is a
backreference followed by literal m. If neither of the
preceding conditions exists, \nm matches octal escape
value nm when n and m are octal digits
(0-7).
|
|
\nml
|
Matches octal escape value nml when n is an
octal digit (0-3) and m and l are octal digits
(0-7).
|
|
\un
|
Matches n, where n is a Unicode character
expressed as four hexadecimal digits. For example, \u00A9 matches
the copyright symbol (©).
|
Order of
Precedence
From Highest to Lowest, the Order of Precedence of the Regular
Expression Operators:
|
Operator(s)
|
Description
|
|
\
|
Escape
|
|
(), (?:), (?=), []
|
Parentheses and Brackets
|
|
*, +, ?, {n}, {n,}, {n,m}
|
Quantifiers
|
|
^, $, \anymetacharacter
|
Anchors and Sequences
|
|
|
|
Alternation
|
Characters have higher precedence than the alternation operator,
which allows 'm|food' to match "m" or "food". To match "mood" or
"food", use parentheses to create a subexpression, which results in
'(m|f)ood'.
Character Matching
The period (.) matches any single printing or non-printing
character in a string, except a newline character (\n). The
following regular expression matches 'aac', 'abc', 'acc', 'adc',
and so on, as well as 'a1c', 'a2c', a-c', and a#c':
If you are trying to match a string containing a word where a
period (.) is part of the input string, you do so by preceding the
period in the regular expression with a backslash (\) character. To
illustrate, the following regular expression matches
'filename.ext':
Bracket Expressions
You can create a list of matching characters by placing one or
more individual characters within square brackets ([ and ]). When
characters are enclosed in brackets, the list is called a
bracket expression. Within brackets, as anywhere else,
ordinary characters represent themselves, that is, they match an
occurrence of themselves in the input text. Most special characters
lose their meaning when they occur inside a bracket expression.
Here are some exceptions:
- The ']' character ends a list if it's not the first item. To
match the ']' character in a list, place it first, immediately
following the opening '['.
- The '\' character continues to be the escape character. To
match the '\' character, use '\\'.
Characters enclosed in a bracket expression match only a single
character for the position in the regular expression where the
bracket expression appears. The following JScript regular
expression matches 'Chapter 1', 'Chapter 2', 'Chapter 3', 'Chapter
4', and 'Chapter 5':
If you want to express the matching characters using a range
instead of the characters themselves, you can separate the
beginning and ending characters in the range using the hyphen (-)
character. The character value of the individual characters
determines their relative order within a range. The following
regular expression contains a range expression that is equivalent
to the bracketed list shown above.
When a range is specified in this manner, both the starting and
ending values are included in the range.
If you want to include the hyphen character in your bracket
expression, you must do one of the following:
- Escape it with a backslash: [\-]
You can also find all the characters not in the list or range by
placing the caret (^) character at the beginning of the list. If
the caret character appears in any other position within the list,
it matches itself, that is, it has no special meaning. The
following regular expression matches chapter headings with numbers
greater than 5':
OR
A typical use of a bracket expression is to specify matches of
any upper- or lowercase alphabetic characters or any digits. The
following regular expression specifies such a match:
Quantifiers and Associated
Meanings
Sometimes, you do not know how many characters there are to
match. In order to accommodate that kind of uncertainty, regular
expressions support the concept of quantifiers. These quantifiers
let you specify how many times a given component of your regular
expression must occur for your match to be true.
|
Character
|
Description
|
|
*
|
Matches the preceding character or subexpression zero or more
times. For example, 'zo*' matches "z" and "zoo". * is equivalent to
{0,}.
|
|
+
|
Matches the preceding character or subexpression one or more
times. For example, 'zo+' matches "zo" and "zoo", but not "z". + is
equivalent to {1,}.
|
|
?
|
Matches the preceding character or subexpression zero or one
time. For example, 'do(es)?' matches the "do" in "do" or "does". ?
is equivalent to {0,1}
|
|
{n}
|
n is a nonnegative integer. Matches exactly n
times. For example, 'o{2}' does not match the 'o' in "Bob," but
matches the two o's in "food".
|
|
{n,}
|
n is a nonnegative integer. Matches at least n
times. For example, 'o{2,}' does not match the 'o' in "Bob" and
matches all the o's in "foooood". 'o{1,}' is equivalent to 'o+'.
'o{0,}' is equivalent to 'o*'.
|
|
{n,m}
|
m and n are nonnegative integers, where
n <= m. Matches at least n and at
most m times. For example, 'o{1,3}' matches the first
three o's in "fooooood". 'o{0,1}' is equivalent to 'o?'. Note that
you cannot put a space between the comma and the numbers.
|
With a large input document, chapter numbers could easily exceed
nine, so you need a way to handle two or three digit chapter
numbers. Quantifiers give you that capability. The following
regular expression matches chapter headings with any number of
digits:
If you know that your chapter numbers are limited to only 99
chapters, you can use the following regular expression to specify
at least one, but not more than 2 digits.
The disadvantage to the expression shown above is that if there
is a chapter number greater than 99, it will still only match the
first two digits. Another disadvantage is that somebody could
create a Chapter 0 and it would match. A better expression for
matching only two digits are the following:
The '*', '+', and '?' quantifiers are all what are referred to
as greedy, that is, they match as
much text as possible. Sometimes that is not at all what
you want to happen. Sometimes, you just want a minimal match.
Say, for example, you are searching an HTML document for an
occurrence of a chapter title enclosed in an H1 tag. That text
appears in your document as:
<H1>Chapter 1 – Introduction to Regular
Expressions</H1>
The following expression matches everything from the opening
less than symbol (<) to the greater than symbol (>) at the
end of the closing H1 tag.
If all you really wanted to match was the opening H1 tag, the
following, non-greedy expression matches only
<H1>.
By placing the '?' after a '*', '+', or '?' quantifier, the
expression is transformed from a greedy to a
non-greedy, or minimal, match.
Anchors
So far, the examples you've seen have been concerned only with
finding chapter headings wherever they occur. Any occurrence of the
string 'Chapter' followed by a space, followed by a number, could
be an actual chapter heading, or it could also be a cross-reference
to another chapter. Since true chapter headings always appear at
the beginning of a line, you'll need to devise a way to find only
the headings and not find the cross-references.
Anchors provide that capability. Anchors allow you to fix a regular
expression to either the beginning or end of a line. They also
allow you to create regular expressions that occur either within a
word or at the beginning or end of a word. The following table
contains the list of regular expression anchors and their
meanings:
|
Character
|
Description
|
|
^
|
Matches the position at the beginning of the input string.
|
|
$
|
Matches the position at the end of the input string.
|
|
\b
|
Matches a word boundary, that is, the position between a word
and a space.
|
|
\B
|
Matches a nonword boundary.
|
To match text at the beginning of a line of text, use the '^'
character at the beginning of the regular expression. Don't confuse
this use of the '^' with the use within a bracket expression.
They're definitely not the same.
To match text at the end of a line of text, use the '$'
character at the end of the regular expression.
To use anchors when searching for chapter headings, the
following regular expression matches a chapter heading with up to
two following digits that occurs at the beginning of a line:
Not only does a true chapter heading occur at the beginning of a
line, it's also the only thing on the line, so it also must be at
the end of a line as well. The following expression ensures that
the match you've specified only matches chapters and not
cross-references. It does so by creating a regular expression that
matches only at the beginning and end of a line of text.
| ^Chapter [1-9][0-9]{0,1}$
|
Matching word boundaries is a little different but adds a very
important capability to regular expressions. A word boundary is the
position between a word and a space. A non-word boundary is any
other position. The following expression matches the first three
characters of the word 'Chapter' because they appear following a
word boundary:
The position of the '\b' operator is critical here. If it's
positioned at the beginning of a string to be matched, it looks for
the match at the beginning of the word; if it's positioned at the
end of the string, it looks for the match at the end of the word.
For example, the following expressions match 'ter' in the word
'Chapter' because it appears before a word boundary:
The following expressions match 'apt' as it occurs in 'Chapter',
but not as it occurs in 'aptitude':
That's because 'apt' occurs on a non-word boundary in the word
'Chapter' but on a word boundary in the word 'aptitude'. For the
non-word boundary operator, position isn't important because the
match isn't relative to the beginning or end of a word.
Alternation and Grouping
Alternation allows use of the '|' character to allow a choice
between two or more alternatives. Expanding the chapter heading
regular expression, you can expand it to cover more than just
chapter headings. However, it's not as straightforward as you might
think. When alternation is used, the largest possible expression on
either side of the '|' character is matched. You might think that
the following expressions match either 'Chapter' or 'Section'
followed by one or two digits occurring at the beginning and ending
of a line:
| ^Chapter|Section
[1-9][0-9]{0,1}$
|
Unfortunately, what happens is that the regular expressions
shown above match either the word 'Chapter' at the beginning of a
line, or 'Section' and whatever numbers follow that, at the end of
the line. If the input string is 'Chapter 22', the expression shown
above only matches the word 'Chapter'. If the input string is
'Section 22', the expression matches 'Section 22'. But that's not
the intent here so there must be a way to make that regular
expression more responsive to what you're trying to do and there
is.
You can use parentheses to limit the scope of the alternation,
that is, make sure that it applies only to the two words, 'Chapter'
and 'Section'. However, parentheses are tricky as well, because
they are also used to create subexpressions, something that's
covered later in the section on subexpressions. By taking the
regular expressions shown above and adding parentheses in the
appropriate places, you can make the regular expression match
either 'Chapter 1' or 'Section 3'.
The following regular expressions use parentheses to group
'Chapter' and 'Section' so the expression works properly:
| ^(Chapter|Section)
[1-9][0-9]{0,1}$
|
See also:
|
