Regular expressions are a very useful tool in a programmer’s toolbox. But they can’t do everything. And one thing they can’t do is reliably parse CSV (comma separated values) files. This is because regular expressions are not state-preserving. A state machine (or equivalent) is required to parse CSV files.
For example, consider the following (very short) CSV file (3 double quotes + 1 comma + 3 double quotes).
“”, “””
This is correctly interpreted as:
Quote to start data value + escaped quote + comma + escaped quote + quote to end data value
For example, the following single value:
“,”
How each character is interpreted depends on what characters come before and after it. For example, the first quote puts you in the “data internal” state. The second quote creates a “The next character may be escaped or may be the end of data” condition. The third quote brings us back to the “data inside” state.
No matter how complex a regular expression you come up with, there’s always a chance that you’ll end up with a CSV file that won’t parse correctly with your regular expression. And if the parse fails, everything after that can be garbage.
You can create regular expressions that can process CSV files that are guaranteed to have no commas, quotation marks, or carriage returns in the data values. However, commas, quotes, or carriage returns within data values are perfectly valid in CSV files. Therefore, it can only process a subset of all well-formed CSV files.
be careful. can Parses a TSV (tab-separated values) file with regular expressions. TSV files (usually!) cannot contain tabs or carriage returns in their data, so they don’t need to be escaped.
See also Stackoverflow.
Parse HTML using regular expressions: Why not?
