Extract YYYY-mm-dd dates from long text

Question

I have a html document with many lines like:

<option value="29">Soil Temperature (<i>2002-10-17</i>)</option>

or like:

<option value="STO">Soil &amp; Air Temperature (2002-10-17)</option>

For each line, I want to check if the line contains the date in the YYYY-mm-dd format and if it does, I want to extract the date.

The following code does not work:

line = '<option value="29">Soil Temperature (<i>2002-10-17</i>)</option>'
date = datetime.datetime.strptime(line, '%Y-%m-%d')

It gives me the error:

ValueError: time data '<option value="29">Soil Temperature (<i>2002-10-17</i>)</option>' does not match format '%Y-%m-%d'

Any way how to easily extract the date?

is `9999-99-99` a date?? – Mehdi Khademloo Oct 11 '14 at 05:05 — Mehdi Khademloo, Oct 11 '14 at 05:05

falsetru · Accepted Answer · 2014-10-12T16:52:36.657

3

You can use the following pattern:

\b\d{4}-\d\d?-\d\d?\b

>>> import datetime
>>> import re
>>>
>>> line = '<option value="29">Soil Temperature (<i>2002-10-17</i>)</option>'
>>> dt_list = re.findall(r'\b\d{4}-\d\d?-\d\d?\b', line)
>>> [datetime.datetime.strptime(dt, '%Y-%m-%d') for dt in dt_list]
[datetime.datetime(2002, 10, 17, 0, 0)]

NOTE: You should escape \ or use raw string literal as shown in the above example. Otherwise it will interpreted as a escape sequence. Especially \b will be interpreted as a BACKSPACE instead of word boundary.

edited Oct 12 '14 at 16:52

answered Oct 11 '14 at 02:36

falsetru

357,413
63
732
636

2

Does not work if month is only one digit, or if day is only one digit. Change d{2} to d{1,2} in both places to fix. – Marichyasana Oct 11 '14 at 02:58
@Marichyasana, Thank you for suggestion. I updated the answer accordingly. – falsetru Oct 11 '14 at 03:00
why is it necessary to use the word boundary \b mark? – jirikadlec2 Oct 11 '14 at 03:33
2

@jirikadlec2, I put word boundary to prevent matching non-date string: for example, `123456-99-888` – falsetru Oct 11 '14 at 03:34
1

you could rewrite `\d{1,2}` as `\d\d?` that might be easier to understand: `r'\b\d\d\d\d-\d\d?-\d\d?\b'` – jfs Oct 12 '14 at 16:48
@J.F.Sebastian, Yes, it's easier to read/understand. I updated the answer accordingly. Thank you for suggestion. – falsetru Oct 12 '14 at 16:53

score 3 · Answer 2 · edited May 23 '17 at 12:28

Alternatively, you can use a BeautifulSoup HTML parser in conjunction with dateutil:

from bs4 import BeautifulSoup
from dateutil.parser import parse


data = """
<select>
    <option value="29">Soil Temperature (<i>2002-10-17</i>)</option>
    <option value="STO">Soil &amp; Air Temperature (2002-10-17)</option>
</select>
"""

soup = BeautifulSoup(data)
for option in soup('option'):
    print parse(option.text, fuzzy=True)

Prints datetime objects:

2002-10-17 00:00:00
2002-10-17 00:00:00

Note that fuzzy parsing has a bit surprising behavior - if the date is not found in the string, it would return the current date - see Trouble in parsing date using dateutil.

Extract YYYY-mm-dd dates from long text

2 Answers2