String Patterns

What are String Patterns?

String patterns are, in essence, just strings. What makes them different from ordinary strings then, you ask? String patterns are strings that use a special combination of characters. These characters combinations are generally used with functions in the string library such as 'string.match' and 'string.gsub' to do interesting things with strings. For instance, with string patterns you can do something like this:

local s = "I am a string!"
for i in string.gmatch(s, "[^%s]+") do --Where "[^%s]+" is the string pattern.
    print(i)
end

Output:
I
am
a
string!

But what makes the code above so cool? Perhaps you've wanted to make a list of people without using a table, or maybe you need to parse a string. String patterns can help do this!

The Basics of String Patterns

As said before, string patterns are strings that look a little different and are used for a different purpose than what strings are usually used for. Here we will look at the basics of just what make a string pattern up. Here we will look at just what the different parts of a string pattern mean.

Character Classes

Character classes in string patterns stand for a range or set of characters. Let's look at the classes listed below.

• %a

• This character class represents all letters no matter if they're lowercase or uppercase.
• Some examples are: 'a', 'd', 'F', and 'G'.

• %l

• This character class represents all lowercase letters.
• Some examples are: 'a', 'd', 'f', and 'g'.

• %u

• This character class represents all uppercase letters.
• Some examples are: 'A', 'B', 'D', and 'Z'.

• %p

• This character class represents all punctuation characters.
• Some examples are: ".", "?", "+", and "/".

• %w

• This character class represents all alphanumeric letters.

• This means that this class encompasses both letters and numbers.

• Some examples are 'A', 'f', '3', and '7'.

• %d

• This class represents all base 10 numbers.
• Examples are '0', '1', '2' all the way up to '9'

• %s

• This character class represents all space characters.
• Some examples are ' ', '\n', and '\r'

• %c

• This character class represents all control characters.
• Control characters are characters with an ASCII code below 32 and also ASCII code 127
• Control characters are all non-printing meaning that they don't represent a symbol representation.

• %x

• This character class represents all hexadecimal (Base 16) characters.
• Some examples are '21' which represents '!' and '5A' which represents 'Z'

• %z

• This character class represents the character '\0'.
• This character is commonly referred to as NUL.

• The dot character class

• This class is represented by a single dot '.'
• This class represents all characters, every single one.
• Unlike the others, it is not preceded by a '%' sign.

As you can see, each of the character classes are used to represent a set of characters. Now let's look at some of the many things we can do with just these character classes.

Classes can also be used to represent a sequence of a type of characters. For instance, %d%l would match a number that is followed by a lowercase letter. Look at the following example:

local s = "abc123"
local Pattern = "%a%a%a%d" --Matches three letters and a digit
print( string.match( s, Pattern ) )

Output:
abc1

One of the things you might notice about the character classes I mentioned above, is that they are all lowercase. Making them capitals reverses their effect. For instance, %s represents spaces, but %S represents everything except space characters. %l represents lowercase letters which %L represents its compliment, all characters except those that are lowercase letters. Let's look at this example:

local s1 = "a4-2" --Letter, not a letter, punctuation, not a letter
local s2 = "aA-2" --Letter, letter, punctuation, not a letter
local Pattern = "%a%A%p%A" --Matches a letter, not a letter, punctuation, and not a letter
print( string.match( s1, Pattern ) )
print( string.match( s2, Pattern ) )

Output:
a4-2
nil

Why did it print out a4-2? It's because that s1 matched the pattern while s2 did not match the pattern.

Pattern Items

Pattern items can be used to make your code simpler. Here are the pattern items and their definitions, we will explain them below.

• a single character class, which matches a single character in the string

• a single character class followed by a '+', which matches 1 or more repetitions in the string. These repetition items will always match the longest possible sequence.

• a single character class followed by a '*' (asterisk), which matches 0 or more repetitions in the string. These repetition items will always match the longest possible sequence.

• a single character class followed by a '-', which matches 0 or more repetitions in the string. These repetitions will always match the shortest possible sequence.

• a single character class followed by a '?', which matches 0 or 1 occurrence of the string.

Now let's look at how to use them. In these examples, we will use the string.match function. Lets say you have a string like this:

local s = "abc1234567efg"

and you want to retrieve the numbers from that string with patterns. One way you could do that is by using the pattern "%d%d%d%d%d%d%d" which would match seven digits in a row. But what happens if you don't know how many digits there are? For this, you can use pattern items, specifically the '+' pattern item for this example.

local s = "abc1234567efg"
local Pattern = "%d+" --See how I used the '+' pattern item to make it shorter?
print( string.match( s, Pattern ) ) 

Output:
1234567

Now how does this work exactly? As we know, a character class followed by a '+' matches one or more repetitions. For this example, it means that it would match the first digits it finds until it reaches the end of the string or a non-digit.

Now let's take a look at the next pattern item '*'. The difference between '+' and the '*' items is that the '+' item matches 1 or more while the '*' item matches 0 or more. This means that if the character class that is followed by this pattern item isn't represented in the string, it doesn't matter because no matches are required.

local s1 = "1,!643"
local s2 = "12349"
local Pattern = "%d%p*%d" --Matches a digit followed by 0 or more punctuation character followed by another digit.
print( string.match( s1, Pattern ) )
print( string.match( s2, Pattern ) )

Output:
1,!6
12

As you can see, it matches a digit, punctuation characters (if there is one), and then another digit. If you had used the '+' item, the second example would have returned nil because that pattern item requires at least one match. The '*' pattern is very useful when you have something in the string that is optional.

Unlike the '*' and '+' pattern items, the '-' item matches the shortest possible sequence. For example, if you have a string that starts and ends with a digit and you want to retrieve a part of the string only up to the second digit in the string, then you can use the '-' item. This example shows you the difference you'd get if you used '-' compared to the '*' item.

local s = "5ab2__0"
local Pattern1 = "%d.-%d" --Matches a digit followed by any character using the shortest possible sequence followed by another digit.
local Pattern2 = "%d.*%d" --The same as the above except it matches the longest possible sequence.
print( string.match( s, Pattern1 ) )
print( string.match( s, Pattern2 ) )

Output:
5ab2
5ab2__0

From the example, you see that using the '-' item found the shortest possible sequence and stopped at the second digit while using the '*' item matched the longest sequence and stopped only at the last digit in the string.

The '?' pattern item is much different than the others because it matches only 0 or 1 occurrence of the string. This is used to make certain characters in the string optional. This makes it a bit like the '*' item except that instead of matching 0 or more occurrences, it only matches 0 or 1.

local s1 = "1.56"
local s2 = "7890"
local s3 = "7..890"
local Pattern = "%d%p?%d"
print( string.match( s1, Pattern ) )
print( string.match( s2, Pattern ) )
print( string.match( s3, Pattern ) )

Output:
1.5
78
89

From the example, you can see, the '?' item matches either 0 or 1. In the first string, there is a single dot between the numbers which this pattern item matches. In the second string, there is no punctuation at all so the punctuation is skipped over. Finally, in the third example, the '?' item matches the first dot, but not the second. In this case, it's skipped over and a match is found imminently after the two dots.

Sets

Sets are used when a single character class cannot do the whole job. For instance, you might want to match both lowercase letters (%l) as well as punctuation characters (%p) using a single class. So how would we do this? Let's take a look at this example:

local s = "123 Hello! I am another string."
local Pattern = "[%l%p]+"
print(string.match(s, Pattern))

Output:
>ello!

As you can see from the example, sets are defined by the '[' and ']' around them. You also see that the classes for lowercase letters and punctuation is contained within. This means that the set will act as a class that represents both lowercase and punctuation, unlike if you used %l%p which would match the sequence of a punctuation character following a lowercase letter.

You aren't restricted to using only character classes, though! You can also use normal characters to add to the set. Also, you can specify a range of characters with the '-' symbol. Let's see how this works in the following example:

local NormCharP = "[3_%l]+" --A set representing lowercase letters, a three, and an underscore that matches 0 or more repetitions.
local RangeP = "[1-4%u]+" --A set representing the numbers 1 to 4 as well as uppercase letters that matches 0 or more repetitions.
local s1 = "Random_123"
local s2 = "37913 Sandwiches!"

for i in string.gmatch(s1, NormCharP) do
    print(i)
end
print("--Next--")
for i in string.gmatch(s2, RangeP) do
    print(i)
end

Output:
andom_
3
--Next--
3
13
S

From the example, you can see how string.gmatch manipulated strings s1 and s2 using the string patterns. And yet, there's still one last thing you can do. Now you can see just how the example in the introduction works. Let's take a look at this code:

local Pattern = "[^%s1-9]+" 
local s = "He29ll0, I like strings1"
local temp = "
for i in string.gmatch(s, Pattern) do
    temp = temp .. i
end
print(temp)

Output:
Hell0,Ilikestrings

This pattern is the compliment of [%s1-9] meaning that it will represent all characters except the space characters and the numbers 1 to 9. This is defined by using the '^' character at the beginning of the set. All it does is makes the set act the direct opposite of a normal set. As you can easily see from this example, the spaces and number 29 in the middle of 'Hello' were removed.

Captures

Captures are used to get pieces of a string that match a capture. Captures are defined by parentheses around them. For instance, (%a%s) is a capture for a letter and a space. When a capture is matched, it is then stored for future use. Let's look at this example:

local s = "TwentyOne = 21"
local Pattern = "(%a+)%s=%s(%d+)"
_, _, key, val = string.find( s, Pattern ) --see how I used parenthesis to designate my captures? "key" is the first capture, and "val" is the second capture.

print( key, val )

Output:
>"TwentyOne 21" --See how it only printed the captures designated by the parenthesis?

Now what happens if you want to get a list by using captures? You can use string.gmatch to do this.

local s = "TwentyOne = 21 Two=2 One =7 Four= 4"
local Pattern = "(%a+)%s?=%s?(%d+)"
for key, val in string.gmatch(s, Pattern) do
    print( key, val )
end

Output:
TwentyOne 21
Two 2
One 7
Four 4

As you can see, string.gmatch iterated through all the matches and returned the captures key and val.

A final thing about captures is that captures can be empty. In these cases they will capture the current position on the string. This means that unlike the other captures, a number is returned instead of a string. Look at this example:

local s = "Hello!"
local Pattern = "()%a+()"
local _, _, cap1, cap2 = string.find( s, Pattern ) )
print( cap1, cap2)

Output:
1 6

As you can see, the empty captures captured the first and last character and returned their positions.

See also

• String Manipulation