Name: APPLICATION OF REGEX IN network SECURITY
Uploaded: 2023-11-09
Duration: 16 min 7 s
Description: [image] abstract image. APPLICATION OF REGEX IN network SECURITY.

[image] abstract image. APPLICATION OF REGEX IN network SECURITY. 

Scene 1

INDEX. [image] group of people at a conference table. 

Scene 2

[image] group of people at a conference table

INTRODUCTION FLAVORS OF REGEX  IMPORTANCE OF REGEX IN NETWORK SECURITY REAL LIFE EXAMPLES  CLOSURE

/REGEX?/. REGULAR EXPRESSIONS. INTRODUCTION. [image] table with various people working on their laptops. 

Scene 3

[image] table with various people working on their laptops

Numerous domains within the realms of computer science and cybersecurity demand the utilization of analytical and technical procedures. To ensure the successful functioning of various processes, it is imperative to cultivate sound logic and analytical abilities. Consequently, regular expressions, commonly referred to as "regex“, find application in tasks such as testing, validation, log parsing, and numerous other technological processing methods. A sequence of characters known as a regular expression plays a pivotal role in identifying specific search patterns. This pattern is employed in tasks related to manipulating text strings, synchronizing data, amalgamating information, as well as facilitating pattern matching in various applications.

WORKING OF REGEX. String with e__...N FORMAT. e matches the literal character “e” [a-z]* matches any run of lowercase letters, and n is a literal match for the letter “n”. 

Scene 4

e matches the literal character “e” [a-z]* matches any run of lowercase letters, and n is a literal match for the letter “n”

^[a-zA-Z0-9._%+-] matches with n number of characters at the beginning without case sensitivity @ matches with the literal special character “@” [a-zA-Z0-9.-] matches any number of characters after “@” without case sensitivity \.[a-zA-Z]$ matches string after “.” that is larger than the length of 2 without case sensitivity

REGEX - /e[a-z]*n/
 eleven – true echo – false erosion – true ear – false tester – false earn – true security – false eruption – true

REGEX - ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]$
 email@gmail.com – true notanemail.com – false abc1764@gmail.com – true someone@stackabuse.ac.in –true qwerty.zzz.abc.com – false false@email – false

Flavors of regex. [image] close up of computer code. 

Scene 5

BASIC REGEX. Basic regular expressions (regex) are a fundamental tool for pattern matching and text manipulation in programming and text processing. Pattern Matching: Regular expressions are a powerful tool for pattern matching in strings. They allow you to define a pattern that describes a set of strings, and then you can use this pattern to search, match, or manipulate text. Metacharacters: Regular expressions use metacharacters like . (matches any character), * (matches zero or more occurrences of the preceding character), + (matches one or more occurrences), ? (matches zero or one occurrence), and more. These metacharacters provide flexibility in defining patterns. Character Classes: Character classes in regex allow you to match specific sets of characters. For example, [0-9] matches any digit, and [A-Za-z] matches any uppercase or lowercase letter. Anchors: Anchors are used to specify the position within the text where a match should occur. ^ anchors to the beginning of a line, and $ anchors to the end of a line. These are often used for exact string matching. Quantifiers: Quantifiers specify how many times a character or group of characters should occur. For example, x matches between 2 and 4 consecutive occurrences of 'x'. Quantifiers include (exactly n occurrences), (at least n occurrences), and (between n and m occurrences).. 

Scene 6

Basic regular expressions (regex) are a fundamental tool for pattern matching and text manipulation in programming and text processing.
 Pattern Matching: Regular expressions are a powerful tool for pattern matching in strings. They allow you to define a pattern that describes a set of strings, and then you can use this pattern to search, match, or manipulate text. Metacharacters: Regular expressions use metacharacters like . (matches any character), * (matches zero or more occurrences of the preceding character), + (matches one or more occurrences), ? (matches zero or one occurrence), and more. These metacharacters provide flexibility in defining patterns. Character Classes: Character classes in regex allow you to match specific sets of characters. For example, [0-9] matches any digit, and [A-Za-z] matches any uppercase or lowercase letter. Anchors: Anchors are used to specify the position within the text where a match should occur. ^ anchors to the beginning of a line, and $ anchors to the end of a line. These are often used for exact string matching. Quantifiers: Quantifiers specify how many times a character or group of characters should occur. For example, x matches between 2 and 4 consecutive occurrences of 'x'. Quantifiers include (exactly n occurrences), (at least n occurrences), and (between n and m occurrences).

Extended Regular Expressions (ERE) are a more feature-rich version of regular expressions that introduce additional metacharacters and capabilities.
 Alternation Operator (|): ERE allows the use of the | (pipe) symbol to specify alternatives. For example, (cat|dog) matches either "cat" or "dog." This feature is not available in basic regular expressions. Grouping with Parentheses: Just like in basic regular expressions, ERE allows grouping with parentheses to apply quantifiers or alternations to a group of characters. For example, (abc) matches "abc" repeated 2 or 3 times. Backreferences: ERE supports backreferences, which allow you to refer to previously captured groups in your expression. For example, ([a-z])\1 matches repeated lowercase letters (e.g., "aa" or "bb"). Non-capturing Groups: ERE introduces non-capturing groups, denoted by (?: ...), which allow you to group expressions without capturing them as a separate match. This can be useful for organizing your regular expression without creating unnecessary capture groups. Word Boundaries: ERE supports the use of word boundaries with \b, which allows you to match patterns at the beginning or end of words. For example, \bword\b would match "word" as a whole word but not within another word like "keyword."

PERL COMPATIBLE REGEX 2 – mostly used. 7. PCRE2 (Perl Compatible Regular Expressions 2) is a library and a set of tools for working with regular expressions in various programming languages. Compatibility with Perl: PCRE2 is designed to be compatible with Perl regular expressions, which means that the regular expressions used in PCRE2 closely resemble those used in the Perl programming language. This compatibility ensures that complex patterns and expressions used in Perl can also be used in PCRE2. Unicode Support: PCRE2 provides robust support for working with Unicode-encoded text. It can handle a wide range of character encodings, making it suitable for internationalization and multilingual text processing. Backtracking and Performance: PCRE2 uses a backtracking approach for pattern matching. While this approach is powerful and flexible, it can have performance implications for very large or complex patterns, as backtracking can be resource-intensive. PCRE2 provides various optimization techniques to improve performance, including Just-In-Time (JIT) compilation. Capture Groups: PCRE2 allows the use of capture groups, which are portions of a regular expression enclosed in parentheses. Capture groups are used to extract specific parts of a matched string, making it easier to process and manipulate text. Programming Language Support: PCRE2 is available as a C library, but it also has bindings for many programming languages, including Python, Ruby, PHP, and more. This broad language support makes it accessible and usable in a wide range of development environments.. 

Scene 7

PCRE2 (Perl Compatible Regular Expressions 2) is a library and a set of tools for working with regular expressions in various programming languages.
 Compatibility with Perl: PCRE2 is designed to be compatible with Perl regular expressions, which means that the regular expressions used in PCRE2 closely resemble those used in the Perl programming language. This compatibility ensures that complex patterns and expressions used in Perl can also be used in PCRE2.
 Unicode Support: PCRE2 provides robust support for working with Unicode-encoded text. It can handle a wide range of character encodings, making it suitable for internationalization and multilingual text processing.
 Backtracking and Performance: PCRE2 uses a backtracking approach for pattern matching. While this approach is powerful and flexible, it can have performance implications for very large or complex patterns, as backtracking can be resource-intensive. PCRE2 provides various optimization techniques to improve performance, including Just-In-Time (JIT) compilation.
 Capture Groups: PCRE2 allows the use of capture groups, which are portions of a regular expression enclosed in parentheses. Capture groups are used to extract specific parts of a matched string, making it easier to process and manipulate text.
 Programming Language Support: PCRE2 is available as a C library, but it also has bindings for many programming languages, including Python, Ruby, PHP, and more. This broad language support makes it accessible and usable in a wide range of development environments.

or
9 or
ö or
or
or
or
or
1 or
I, greedy
1, possessive
I, lazy
more ,
more ,
more ,
more ,
more ,
more ,
exactly n
at least n,
at Least n,
at Least n,
n or more,
n or more,
n or more,
greedy
possessa ve
lazy
greedy
possess-I ve
lazy
no more than m,
no more than m,
no more than m,
greedy
possessive
lazy
greedy
lazy

[image] close up of computer code. AN OVERVIEW. IMPORTANCE OF REGEX IN NETWORK SECURITY. 

Scene 8

NEED FOR REGEX. Threat Detection and Alerting Log Analysis and Incidence Response Correlation and Contextualization Anomaly Detection Customized Reporting Reduction of Log Noise, etc.. 

Scene 9

Threat Detection and Alerting Log Analysis and Incidence Response Correlation and Contextualization Anomaly Detection Customized Reporting Reduction of Log Noise, etc.

THREAT DETECTION AND ALERTING. 1. Identify Threat Patterns: Define the specific threat indicators you want to detect, such as IP addresses, URLs, or SQL injection attempts. 2. Create Regex Patterns: Develop regular expressions (regex) that match the identified threat patterns. 3. Implement Regex Matching: Use programming languages or tools to apply regex patterns to your data, like Python or grep. 4. Alerting and Response: Trigger alerts or take predefined actions when regex patterns match, following your response plan. 5. Tuning and Maintenance: Regularly update regex patterns to adapt to evolving threats and reduce false positives. 6. Scalability: Optimize performance for large-scale systems when processing significant amounts of data.. 

Scene 10

1. Identify Threat Patterns:  Define the specific threat indicators you want to detect, such as IP addresses, URLs, or SQL injection attempts. 2. Create Regex Patterns:  Develop regular expressions (regex) that match the identified threat patterns. 3. Implement Regex Matching:  Use programming languages or tools to apply regex patterns to your data, like Python or grep. 4. Alerting and Response:  Trigger alerts or take predefined actions when regex patterns match, following your response plan. 5. Tuning and Maintenance:  Regularly update regex patterns to adapt to evolving threats and reduce false positives. 6. Scalability:  Optimize performance for large-scale systems when processing significant amounts of data.

threatdet.py
import re
threat_patterns
r' (httplhttps) : // [a-zA-Ze-9. -]+' ,
# IP address pattern
# URL pattern
data = "An alert was triggered for IP address 192.168.1.1 and URL http://example.com"
for pattern in threat_patterns:
— re . findall(pattern, data)
matches —
if matches:
print(f"A1ert: Found potential threat pattern
'
in the data:" )

FIG 1. Sample python program for threat detection

LOG ANALYSIS AND INCIDENCE RESPONSE. 11. Using regular expressions (regex) for log analysis and incident response is a critical part of many cybersecurity and monitoring workflows. It allows you to search for specific patterns and events in log files and, upon detection, trigger incident response actions. 1. Log Collection: Gather log data from various sources, such as system logs, network logs, application logs, and security logs. 2. Identify Important Log Entries: Determine which log entries or patterns you want to monitor for potential security incidents. These could include login failures, unauthorized access attempts, or other suspicious activities. 3. Create Regex Patterns: Develop regex patterns that match the log entries associated with security incidents. For instance, you might create regex patterns for IP addresses, usernames, or error messages. 4. Log Analysis: Apply the regex patterns to the collected log data. This can be done using log analysis tools, scripting, or custom software. 5. Incident Detection: When the regex patterns match log entries, it signals a potential security incident. Alerts or notifications can be triggered at this point.. 

Scene 11

Using regular expressions (regex) for log analysis and incident response is a critical part of many cybersecurity and monitoring workflows. It allows you to search for specific patterns and events in log files and, upon detection, trigger incident response actions.
 1. Log Collection: Gather log data from various sources, such as system logs, network logs, application logs, and security logs. 2. Identify Important Log Entries: Determine which log entries or patterns you want to monitor for potential security incidents. These could include login failures, unauthorized access attempts, or other suspicious activities. 3. Create Regex Patterns: Develop regex patterns that match the log entries associated with security incidents. For instance, you might create regex patterns for IP addresses, usernames, or error messages. 4. Log Analysis: Apply the regex patterns to the collected log data. This can be done using log analysis tools, scripting, or custom software. 5. Incident Detection: When the regex patterns match log entries, it signals a potential security incident. Alerts or notifications can be triggered at this point.

6. Incident Response: Define a response plan for different types of incidents. This plan can include actions like blocking IP addresses, logging the incident, or notifying the appropriate personnel. 7. Automated Response: In some cases, you can automate response actions based on the regex matches. For example, you might configure an automated firewall rule to block an IP address that is repeatedly attempting unauthorized access. 8. Human Intervention: More complex incidents may require human intervention. In these cases, the system can alert security personnel to investigate and respond to the incident. 9. Review and Improve: Continuously monitor and review your regex patterns and incident response processes. Make adjustments to patterns to reduce false positives and false negatives. 10. Documentation: Properly document all incident response actions and their outcomes for auditing and future reference.
 Using regex for log analysis and incident response helps organizations quickly identify and respond to potential security threats and breaches, which is critical for maintaining the security of computer systems and networks.

CORRELATION AND CONTEXTUALIZATION. Correlation and conceptualization in the context of log analysis, incident response, and security involve understanding relationships between different log entries, events, or data points. While regular expressions (regex) can help in pattern matching, they have limitations in providing a complete picture of complex relationships. However, you can use regex in combination with other techniques to aid in correlation and conceptualization. 1. Correlation: Event Correlation: Regex can be used to identify common patterns in log entries that are associated with a single event. For example, you might use regex to extract and match timestamps or specific event IDs in log entries to determine when an incident occurred. Linking Logs: Regex can help you extract relevant information from log entries and link them to other related log entries. For example, you can use regex to extract a session ID or IP address and then search for other log entries associated with the same session or IP address. Time-Based Correlation: Regex can assist in extracting timestamps from log entries and then analyzing events that occur within a specific time window. This is useful for identifying coordinated attacks or sequences of events.. 

Scene 12

Correlation and conceptualization in the context of log analysis, incident response, and security involve understanding relationships between different log entries, events, or data points. While regular expressions (regex) can help in pattern matching, they have limitations in providing a complete picture of complex relationships. However, you can use regex in combination with other techniques to aid in correlation and conceptualization.  1. Correlation: Event Correlation: Regex can be used to identify common patterns in log entries that are associated with a single event. For example, you might use regex to extract and match timestamps or specific event IDs in log entries to determine when an incident occurred. Linking Logs: Regex can help you extract relevant information from log entries and link them to other related log entries. For example, you can use regex to extract a session ID or IP address and then search for other log entries associated with the same session or IP address. Time-Based Correlation: Regex can assist in extracting timestamps from log entries and then analyzing events that occur within a specific time window. This is useful for identifying coordinated attacks or sequences of events.

2. Conceptualization:
 Pattern Recognition: Regex is a valuable tool for identifying common patterns within log entries, which can help you conceptualize and categorize different types of events. For instance, you might use regex to classify log entries into categories like login attempts, SQL injection attempts, or system errors.
 Taxonomy and Tagging: You can use regex to tag or categorize log entries based on the patterns you identify. This helps in building a taxonomy of events, making it easier to conceptualize and understand the types of activities occurring in your systems.
 Visualization: While regex is not directly used for visualization, once you've categorized and conceptualized log entries, you can use visualization tools to represent relationships and patterns more clearly, providing a visual understanding of your data.
 In practice, log analysis and security information and event management (SIEM) tools often combine regex with other techniques, such as statistical analysis, machine learning, and data correlation engines, to provide a more comprehensive view of the security landscape. Regex is typically one piece of the larger puzzle when it comes to understanding, correlating, and conceptualizing events in a complex and dynamic environment.

ANOMALY DETECTION. Anomaly detection using regular expressions (regex) is not a common approach. Anomaly detection typically relies on statistical methods, machine learning, or other data analysis techniques to identify patterns or behaviors that deviate significantly from the norm. However, regex can still play a limited role in some scenarios. Here's how regex might be used in conjunction with other techniques for anomaly detection: 1.Preprocessing and Feature Extraction : Regex can be used to preprocess and extract relevant features from the data before applying more advanced anomaly detection methods. For example, in network traffic analysis, you can use regex to extract IP addresses, URLs, or specific patterns of interest. 2. Pattern-Based Detection : Regex can help identify known patterns that might indicate anomalies. For example, if you have a known pattern for valid email addresses, you can use regex to detect any email addresses in your data that do not match that pattern. 3. Rule-Based Anomaly Detection : Create regex-based rules to detect specific anomalies. For instance, you could use regex to identify unexpected HTTP response codes in web server logs, which might indicate an issue or attack.. 

Scene 13

Anomaly detection using regular expressions (regex) is not a common approach. Anomaly detection typically relies on statistical methods, machine learning, or other data analysis techniques to identify patterns or behaviors that deviate significantly from the norm. However, regex can still play a limited role in some scenarios. Here's how regex might be used in conjunction with other techniques for anomaly detection: 1.Preprocessing and Feature Extraction : Regex can be used to preprocess and extract relevant features from the data before applying more advanced anomaly detection methods. For example, in network traffic analysis, you can use regex to extract IP addresses, URLs, or specific patterns of interest. 2. Pattern-Based Detection : Regex can help identify known patterns that might indicate anomalies. For example, if you have a known pattern for valid email addresses, you can use regex to detect any email addresses in your data that do not match that pattern. 3. Rule-Based Anomaly Detection : Create regex-based rules to detect specific anomalies. For instance, you could use regex to identify unexpected HTTP response codes in web server logs, which might indicate an issue or attack.

4. Data Cleaning and Normalization : Regex can assist in cleaning and normalizing data. Removing or transforming noisy or inconsistent data can help improve the accuracy of subsequent anomaly detection methods. 5. Hybrid Approaches : You can combine regex with other anomaly detection techniques. For example, use regex to filter and preprocess log data, and then apply machine learning algorithms to detect anomalies in the processed data.
 It's important to note that regex alone is not suitable for complex anomaly detection tasks because it lacks the ability to learn from historical data or adapt to changing patterns. For more sophisticated anomaly detection, consider leveraging machine learning algorithms such as clustering, classification, or time series analysis, as well as statistical methods like z-scores, moving averages, or exponential smoothing. In summary, while regex can be a useful tool for data preprocessing and basic pattern matching, it is typically just one component of a more comprehensive anomaly detection system that uses more advanced techniques to identify unexpected or suspicious behaviors in data.

CUSTOMIZED REPORTING. Customized reporting using regular expressions (regex) typically involves extracting specific data or patterns from a larger dataset and presenting that data in a structured and readable format. This can be useful for creating tailored reports or summaries from unstructured or semi-structured data sources.. 

Scene 14

Customized reporting using regular expressions (regex) typically involves extracting specific data or patterns from a larger dataset and presenting that data in a structured and readable format. This can be useful for creating tailored reports or summaries from unstructured or semi-structured data sources.

Reducing log noise using regular expressions (regex) is a common practice in log analysis and monitoring to filter out irrelevant or repetitive log entries, leaving behind only the essential information. Here are some common ways to achieve this using regex: Filtering by Log Severity : You can use regex to filter log entries based on their severity level. For example, if log entries include severity indicators like "INFO," "WARNING," or "ERROR," you can create regex patterns to match and retain only the logs of interest. Pattern-Based Filtering : You can use regex to filter out logs that contain specific patterns you want to ignore. For instance, if you want to filter out logs related to known, benign activities, you can create regex patterns that match those patterns and exclude them. Timestamp Filtering : Regex can be used to filter logs based on timestamps. For example, you might only want logs from a specific time frame, and you can create regex patterns to extract and compare timestamps to filter the logs accordingly.

15. REDUCTION OF LOG NOISE. Log Entry Deduplication:Use regex to identify and remove duplicate log entries. If you have log entries that are repeated multiple times, regex can help you detect and remove duplicates. Custom Log Entry Patterns:Sometimes, logs may have custom patterns that you want to filter. For example, if you want to filter logs that contain specific user IDs, URLs, or IP addresses, you can use regex patterns to match and filter logs based on these patterns. Conditional Filtering:Regex can also be used for conditional filtering, such as filtering out logs if they contain both specific patterns or if they meet certain criteria. While regex can be effective in reducing log noise, it's essential to exercise caution and regularly review your filtering patterns to avoid filtering out critical information accidentally. Additionally, for more advanced log noise reduction and analysis, consider using log management and SIEM (Security Information and Event Management) tools that provide more sophisticated log filtering and analysis capabilities.. 

Scene 15

Log Entry Deduplication:Use regex to identify and remove duplicate log entries. If you have log entries that are repeated multiple times, regex can help you detect and remove duplicates. Custom Log Entry Patterns:Sometimes, logs may have custom patterns that you want to filter. For example, if you want to filter logs that contain specific user IDs, URLs, or IP addresses, you can use regex patterns to match and filter logs based on these patterns. Conditional Filtering:Regex can also be used for conditional filtering, such as filtering out logs if they contain both specific patterns or if they meet certain criteria. While regex can be effective in reducing log noise, it's essential to exercise caution and regularly review your filtering patterns to avoid filtering out critical information accidentally. Additionally, for more advanced log noise reduction and analysis, consider using log management and SIEM (Security Information and Event Management) tools that provide more sophisticated log filtering and analysis capabilities.

REAL LIFE EXAMPLES. [image] close up of computer on top of table against a brick wall. 

Scene 16

[image] close up of computer on top of table against a brick wall

Analysis of malicious emails from the log. Every system generates an event or log. Based on the particular log source and the log generated, we can analyze the type of log to determine if it is malicious nor not. For example: If we have various logs that contain a field ‘email’ in every log, we can write the following regex: email=\s(?<object>(?<login>[^@]+)@(?<email_type>[^\.]+)(?<domain>[^\s]+)). 

Scene 17

Analysis of malicious emails from the log

Every system generates an event or log. Based on the particular log source and the log generated, we can analyze the type of log to determine if it is malicious nor not.
 For example: If we have various logs that contain a field ‘email’ in every log, we can write the following regex: email=\s(?<object>(?<login>[^@]+)@(?<email_type>[^\.]+)(?<domain>[^\s]+))

LogMsg
ernatl= abc_xyz@deerwalk edu np
email- hacker@yahoo.com.np
email— test 1101@gmail.com
Logn
abc_xyz
hacker
testl 101
Domal nlmpacted
.edu.np
.com.np
Obgect
abc_xyz@deerwalk edu.np
hacker@yahoo.com.np
testl 101@gmail.com

Parsing important fields from the log source. 18. 

Scene 18

Parsing important fields from the log source

In security engineering, major fields need to be identified that need to be parsed and analyzed. In this case, the type of log, username, logged in or logged out, and source IP address are parsed using the regex:
 \d-\d-\d\s\d:\d:\d\s-\s(?<desc_type>[^:]+):\sUser\s(?<login>[^\s+]+)\s(?<action_type>(logged in|logged out)).*IP\s(?<sip>\d+.\d+.\d.+)

REGULAR EXPRESSION
\ s (logged ' i nitogged ' out)) .
TEST STRING
2023-08-08'15: 35 'User'
5 matches (342 steps, 04ms)
2
2923-68-98-15.
2023-08-08 •
15.
2023-08-08•15 :
• 45 INFO: 'User • Ram. togged. in. from. 192.168.1.10-
Hari from 'IP' 16.161.15.12
Sita • togged •in • from• IP •68.18.2.10
52:19. .
- INFO: •user•
: User'Hacker'togged• in•from'IP'226.138.89.5
Tester • logged• in • from• IP• 27.18.17.10

Here, (1) is the regular expression that parsed the logs present in (2). 2023–08–08 15:30:45 — INFO: User Ram logged in from IP 192.168.1.10 INFO | Ram | logged in | 192.168.1.10 2023–08–08 15:35:46 — INFO: User Hari logged in from IP 10.161.15.12 INFO | Hari | logged in | 10.161.15.12 2023–08–08 15:45:22 — WARN: User Sita logged in from IP 68.18.2.10 WARN | Sita | logged in | 68.18.2.10 2023–08–08 15:47:49 — MALICIOUS: User Hacker logged in from IP 226.138.89.5 MALICIOUS | Hacker | logged in | 226.138.89.5 2023–08–08 15:52:19 — INFO: User Tester logged in from IP 27.18.17.10
 By analyzing the fields and source IP addresses, we can classify the nature of the IP addresses and determine if they are from a malicious source or not.

Identification of brute force attempt to login from multiple IP addresses at the same time. 

Scene 19

Identification of brute force attempt to login from multiple IP addresses at the same time

A brute force attack is a hacking method that uses trial and error to crack passwords, login credentials, and encryption keys. In this method, the attacker tries to login to the victim’s account by using multiple common passwords or a combination of words. For example, if the WPS pin of the router is 67871230, then the attacker might brute force the pin from 00000000 to 99999999. Similarly, if an attacker tries to get a victim credential with multiple passwords from multiple IP addresses, we can notice multiple authentication failure attempts. So, these types of logs can be identified, normalized and parsed using regex.

Failed password for (?<user>\w+) from (?<sip>\d+.\d+.\d+.\d+)\sport\s(?<dport>\d+) Here, (1) is the regular expression, (2) is the group of source IP addresses from which the attacker is attempting a brute force attack, and (3) is the group of various logs generated by the system. In this example, user John is the victim, and the attacker is attempting brute force on his account. We can analyze that the attacker is attacking using multiple IP addresses at the same time on port 80 for user John’s account.

Detection of SQL Injection attempts in web server logs. 

Scene 20

Detection of SQL Injection attempts in web server logs

A type of cybersecurity vulnerability known as SQL injection happens when an attacker is able to change a web application’s or database query’s input parameters such that it executes unauthorized SQL instructions. As a defensive security analyst, we must know SQL syntax, queries, components, and statements. If we can identify the SQL queries in the search field, it might be a SQL injection attack attempt. A demonstration of SQL Injection detection using regex is given below:

REGULAR EXPRESSION
TEST STRING
teeee]
127.0 e.l•-•-• [e9/Aug/2e23
127 e e [eg/Aug/2e23
: 15:45
127 e
teoee]
/ gm
127.9.
127.0. .1
127.0.
127.e.e.1
127.
127.0.
127.e.e.1
t09/Aug/2e23
e • — [e9/Aug/2e23
0.1.-.-. [09/Aug/2e23
[eg/Aug/2e23.
e t09/Aug/2e23
0.1•-•-• [e9/Aug/2e23
[e9/Aug/2e23.
15
: 15
: 15
: 45
: 45
: 22 • • "GET•
- "GEL /search?q-1X20SELECTX2Ø1-1 •HTTP/1 1"
: 24 • •ewe] • "GET • • HTTP/I. 1" • 2%.5678-
"GET- /search?q-h0UNTEVEREST HTTP/I.I" -2ee
• "GET. HTTP/1
: 27 - +eeee] - "GET- HTTP/I. 1" -28B.567b
567b
- "GET- HTTP/1 1"
. -2ee•5678e
5678-
. • 283-5678'
: 28 -
• •eoee] • "GET • •HTTP/ 1.1"
"GET - DHTTP/I.1nD2ee„5678
2

Here (1) is the regular expression and (2) are the logs. We can analyze the logs that contain SQL queries and mark them as suspicious using regex. Here, SQL queries that contain SQL commands such as OR, SELECT, UPDATE, DELETE, UNION, ORDERBY, and AND are identified and marked suspicious.

.*”(?<REQUESTMETHOD>[^\s]+).*(?<SQLINJECTION_CHECK>(OR|SELECT|UPDATE|DELETE|UNION|ORDERBY|AND)\S+).* The above-mentioned regular expression identifies the logs that contains SQL commands and ignores the regular search logs. Due to its flexible capacity to shape, filter, and interpret complex patterns inside data streams, regex (regular expressions) serves as a vital asset for security experts in the field of cybersecurity. Regex allows security engineers to quickly spot anomalies or threats in log files, bolster systems against vulnerabilities through input validation, and uncover hidden patterns in data that could indicate malicious intent.

Summary. [image] person staring at blueprints on a brick wall. 

Scene 21

[image] person staring at blueprints on a brick wall

In a nutshell, regex is a valuable tool that security professionals can use to proactively identify, stop, and mitigate various security risks across a variety of settings with the aim of protecting digital environments. This increases the general resilience of the digital realm.

THANK YOU. [image] Headshot Icon. UDAYAN MATE. 22BCE1764. 