Regular expressions: search for patterns in text

Imagine you are looking for a needle in a haystack. Now imagine that you have a magnet that instantly finds all the needles of a certain shape. This is how regular expressions work — a powerful tool for finding and processing text patterns that saves developers hundreds of hours of routine work.

What are regular expressions

Regular expressions (regex or regexp) are a formal language for describing text patterns. They allow you to find, extract, and replace text according to complex rules using a compact syntax. Instead of writing dozens of lines of code to validate an email address, you can use a single regular expression.

Regular expressions appeared in the 1950s thanks to the mathematician Stephen Kleene, but they became really popular with the development of Unix systems and text editors. Today they are supported in almost all programming languages.

Basic syntax

Let's start with the simple. The most basic regular expression is a regular string. For example, the cat pattern will find all occurrences of the word "cat" in the text. But the real power of regular expressions is revealed through special characters.

Metasymbols are the basis of regex:

. — any character except for a line break. The c.t pattern will find "cat", "cut", "c9t" and so on.

^ — the beginning of the line. The expression ^Hello will find "Hello" only at the beginning of the line.

$ — end of line. The world$ pattern will find "world" only at the end.

* — zero or more repetitions of the previous character. For example, go*gle will find "ggle", "gogle", "google", "goooogle".

+ — one or more repetitions. The go+gle pattern will find "gogle", "google", but not "ggle".

? — zero or one occurrence. The expression colou?r will find both "color" and "colour".

[] is a character class. The [aeiou] pattern will find any vowel, and [0-9] will find any number.

| — logical OR. The expression cat|dog will find either "cat" or "dog".

() — grouping. Brackets allow you to create subexpressions and capture the desired parts of the text.

Practical examples

Let's look at the real challenges developers face.

Email address validation:

import re

def validate_email(email):
    pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}

This expression checks the basic structure of the email: characters before @, domain name, and zone.

Extracting phone numbers:

text = "Contacts: +7(999)123-45-67, 8-800-555-35-35"
pattern = r'[\+\d][\d\-\(\)]{9,}'
phones = re.findall(pattern, text)
print(phones)  # ['+7(999)123-45-67', '8-800-555-35-35']

Replacement of sensitive data:

text = "My card number: 1234-5678-9012-3456"
pattern = r'\d{4}-\d{4}-\d{4}-\d{4}'
masked = re.sub(pattern, '****-****-****-****', text)
print(masked)  # My card number: **** - **** - **** - ****

URL parsing:

url = "https://example.com:8080/path/to/page?param=value#section"
pattern = r'(?P<protocol>https?://)?(?P<domain>[^:/]+)(?::(?P<port>\d+))?(?P<path>/[^?#]*)?(?:\?(?P<query>[^#]*))?(?:#(?P<fragment>.*))?'

match = re.match(pattern, url)
print(match.group('domain'))  # example.com
print(match.group('port'))    # 8080

Advanced techniques

Lookahead and lookbehind are powerful tools for contextual search without text capture.

Positive lookahead (?=...) — checks that a certain pattern follows:

# Find words followed by an exclamation mark
pattern = r'\w+(?=!)'
text = "Hi! How are you? Great!"
print(re.findall(pattern, text))  # ['Hello', 'Great']]

Negative lookahead (?!...) — checks that the pattern does NOT go further:

# Find numbers that are not followed by the ruble sign
pattern = r'\d+(?!₽)'
text = "100₽, 200, 300₽, 400"
print(re.findall(pattern, text))  # ['200', '400']

Greedy and lazy quantifiers:

By default, quantifiers are greedy — they capture the maximum amount of text:

text = "<div>First</div><div>Second</div>"
pattern = r'<div>.*</div>'
print(re.findall(pattern, text))  
# ['<div>First</div><div>Second</div>']  # Captured everything!

By adding ?, we make the quantifier lazy:

pattern = r'<div>.*?</div>'
print(re.findall(pattern, text))  
# ['<div>First</div>', '<div>Second</div>']  # Two separate matches

Named groups make the code readable:

log = "2024-11-25 14:30:15 ERROR Database connection failed"
pattern = r'(?P<date>\d{4}-\d{2}-\d{2}) (?P<time>\d{2}:\d{2}:\d{2}) (?P<level>\w+) (?P<message>.*)'

match = re.match(pattern, log)
print(match.group('level'))   # ERROR
print(match.group('message')) # Database connection failed

Performance and optimization

Regular expressions can be slow if used incorrectly. Here are some tips:

Compile patterns for multiple use:

# Bad — compiles every time
for text in texts:
    if re.match(r'\d{3}-\d{3}', text):
        process(text)

# Good — compiles once
pattern = re.compile(r'\d{3}-\d{3}')
for text in texts:
    if pattern.match(text):
        process(text)

Avoid catastrophic backtracking:

Some patterns can cause exponential growth in execution time:

# Dangerous!
pattern = r'(a+)+'
text = "aaaaaaaaaaaaaaaaaaaaaaaX"  # May freeze

This is due to multiple matching paths on failure. The solution is to use atomic groups or possessive quantifiers where they are supported.

Use anchors:

# Slower
pattern = r'\d{4}-\d{2}-\d{2}'

# Faster if the date is at the beginning of the line
pattern = r'^\d{4}-\d{2}-\d{2}'

Debugging regular expressions

Regex can be difficult to understand. Use visualization and testing tools:

regex101.com — an excellent online tool with explanations

debuggex.com - visualization of patterns in the form of diagrams

regexr.com - interactive sandbox with hints

For complex expressions, add comments with the re.VERBOSE flag:

pattern = re.compile(r'''
    ^                   # Start of line
    (?P<username>       # Group for username
        [a-zA-Z0-9_]{3,16}  # From 3 to 16 alphanumeric characters
    )
    @                   # Symbol @
    (?P<domain>         # Group for domain
        [a-zA-Z0-9.-]+      # Domain name
        \.[a-zA-Z]{2,}      # Point and zone
    )
    $                   # End of line
'

Alternatives to regular expressions

Regex is not always the best solution. For some tasks, it is better to use specialized tools:

For HTML parsing, use BeautifulSoup or lxml instead of regex — HTML is not a regular language, and regex can give unpredictable results.

Use built-in libraries for JSON parsing. For complex grammars, consider parser generators such as PLY or ANTLR.

For simple substring searches, string methods are often faster and clearer than regular expressions.

Conclusion

Regular expressions are a powerful tool in the developer's arsenal. They save time, make the code compact and solve problems that would otherwise require dozens of lines of code. Start with simple patterns, practice on real tasks, and gradually you will master this language of patterns. Remember about performance, use debugging tools and do not be afraid to experiment. Over time, regular expressions will become a natural part of your workflow.

Do you want to get a deeper understanding of programming and learn how to apply tools such as regular expressions in practice? The application Code offers structured courses in Python, JavaScript, and web development for beginner developers. Join our Telegram channel, where you will find useful materials, analysis of complex topics and support from a community of like-minded people ready to help you on your programming journey.