How can I convert a script for collecting data from a single URL to scrape hundreds of URLs from the same domain?

Question

I have successfully figured out how to scrape statistical baseball data for a URL using BeautifulSoup. Example: I can scrape from https://www.baseball-reference.com/players/p/puckeki01.shtml for statistics for baseball player Kirby Puckett.

However, I want to (1) scrape data from hundreds of URLs from this a domain like this and then (2) store the data to analysis (in a single dataframe? in a csv?). But I can't figure out what I need to do to change my code to collect the data for a list of hundreds of URLs from this domain. I have managed to create a list of all of the URLs with their extensions that I need. Here is my code for one player:

url_stats_by_season_kirby_puckett = 'https://www.baseball-reference.com/players/p/puckeki01.shtml'

# get the html
html = urlopen(url_stats_by_season_kirby_puckett)

# create the BeautifulSoup object
soup = BeautifulSoup(html, "lxml")

column_headers_kirby_puckett = [th.getText() for th in soup.findAll('tr', limit=2)[0].findAll('th')]

table_rows_kirby_puckett = soup.select("#batting_standard tr")[0:] 

def extract_player_data(table_rows_kirby_puckett):

    # create the empty list to store the player data
    player_data = []

    for row in table_rows_kirby_puckett:  # for each row do the following

        # Get the text for each table data (td) element in the row
        # Some player names end with ' HOF', if they do, get the text excluding
        # those last 4 characters,
        # otherwise get all the text data from the table data
        player_date = [th.get_text() for th in row.find_all("th")]
        player_list = [td.get_text() for td in row.find_all("td")]
        # there are some empty table rows, which are the repeated 
        # column headers in the table
        # we skip over those rows and and continue the for loop
        if not player_list:
            continue

        # The data we want from the dictionary can be extracted using the
        # player's name, which returns us their pfr url, and "College Stats"
        # which returns us their college stats page

        # add the link associated to the player's pro-football-reference page, 
        # or en empty string if there is no link

        # add the link for the player's college stats or an empty string
        # if ther is no link

        # Now append the data to list of data
        player_data.append(player_date + player_list)

    return player_data

data_kirby_puckett = extract_player_data(table_rows_kirby_puckett)

df_kirby_puckett = pd.DataFrame(data_kirby_puckett)

df_kirby_puckett.loc[15,1:] = df_kirby_puckett.loc[15,1:].shift(3)

df_kirby_puckett.loc[16,1:] = df_kirby_puckett.loc[16,1:].shift(3)

df_kirby_puckett.columns = column_headers_kirby_puckett

Edit below:

def get_kirby_puckett_stats(url_stats, qb_stats_csv):
    # get the html

    for url in url_stats:
        html = urlopen(url)

    # create the BeautifulSoup object
        soup = BeautifulSoup(html, "lxml")

        column_headers_kirby_puckett = [th.getText() for th in soup.findAll('tr', limit=2)[0].findAll('th')]


        table_rows_kirby_puckett = soup.select("#batting_standard tr")[0:] 



# create the empty list to store the player data


        player_data = []
        for row in table_rows_kirby_puckett:  # for each row do the following

            # Get the text for each table data (td) element in the row
            # Some player names end with ' HOF', if they do, get the text excluding
            # those last 4 characters,
            # otherwise get all the text data from the table data
            player_date = [th.get_text() for th in row.find_all("th")]
            player_list = [td.get_text() for td in row.find_all("td")]
            # there are some empty table rows, which are the repeated 
            # column headers in the table
            # we skip over those rows and and continue the for loop
            #if not player_list:
                #continue

        # Extracting the player links
        # Instead of a list we create a dictionary, this way we can easily
        # match the player name with their pfr url
        # For all "a" elements in the row, get the text
        # NOTE: Same " HOF" text issue as the player_list above
        #links_dict = {(link.get_text()) : link["href"] for link in row.find_all("a", href=True)}

        # The data we want from the dictionary can be extracted using the
        # player's name, which returns us their pfr url, and "College Stats"
        # which returns us their college stats page

        # add the link associated to the player's pro-football-reference page, 
        # or en empty string if there is no link

        # add the link for the player's college stats or an empty string
        # if ther is no link

        # Now append the data to list of data
            player_data.append(player_date + player_list)
        #name_code = url_stats - 'https://www.baseball-reference.com/players/'
            with open(qb_stats_csv, "a") as f:
                writer = csv.writer(f)
                writer.writerows(player_data)

Example output for one player:

Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
1990,30,MIN,AL,146,615,551,82,164,40,3,12,80,5,4,57,73,.298,.365,.446,.811,121,246,15,3,1,3,11,*8/79D456,AS
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
1990,30,MIN,AL,146,615,551,82,164,40,3,12,80,5,4,57,73,.298,.365,.446,.811,121,246,15,3,1,3,11,*8/79D456,AS
1991,31,MIN,AL,152,661,611,92,195,29,6,15,89,11,5,31,78,.319,.352,.460,.812,119,281,27,4,8,7,4,*89,"AS,MVP-7,GG"
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
1990,30,MIN,AL,146,615,551,82,164,40,3,12,80,5,4,57,73,.298,.365,.446,.811,121,246,15,3,1,3,11,*8/79D456,AS
1991,31,MIN,AL,152,661,611,92,195,29,6,15,89,11,5,31,78,.319,.352,.460,.812,119,281,27,4,8,7,4,*89,"AS,MVP-7,GG"
1992,32,MIN,AL,160,696,639,104,210,38,4,19,110,17,7,44,97,.329,.374,.490,.864,139,313,17,6,1,6,13,*8/D456,"AS,MVP-2,GG,SS"
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
1990,30,MIN,AL,146,615,551,82,164,40,3,12,80,5,4,57,73,.298,.365,.446,.811,121,246,15,3,1,3,11,*8/79D456,AS
1991,31,MIN,AL,152,661,611,92,195,29,6,15,89,11,5,31,78,.319,.352,.460,.812,119,281,27,4,8,7,4,*89,"AS,MVP-7,GG"
1992,32,MIN,AL,160,696,639,104,210,38,4,19,110,17,7,44,97,.329,.374,.490,.864,139,313,17,6,1,6,13,*8/D456,"AS,MVP-2,GG,SS"
1993,33,MIN,AL,156,682,622,89,184,39,3,22,89,8,6,47,93,.296,.349,.474,.824,120,295,15,7,1,5,7,*89D/7,AS
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
1990,30,MIN,AL,146,615,551,82,164,40,3,12,80,5,4,57,73,.298,.365,.446,.811,121,246,15,3,1,3,11,*8/79D456,AS
1991,31,MIN,AL,152,661,611,92,195,29,6,15,89,11,5,31,78,.319,.352,.460,.812,119,281,27,4,8,7,4,*89,"AS,MVP-7,GG"
1992,32,MIN,AL,160,696,639,104,210,38,4,19,110,17,7,44,97,.329,.374,.490,.864,139,313,17,6,1,6,13,*8/D456,"AS,MVP-2,GG,SS"
1993,33,MIN,AL,156,682,622,89,184,39,3,22,89,8,6,47,93,.296,.349,.474,.824,120,295,15,7,1,5,7,*89D/7,AS
1994,34,MIN,AL,108,482,439,79,139,32,3,20,112,6,3,28,47,.317,.362,.540,.902,129,237,11,7,1,7,7,*9D/8,"AS,MVP-7,SS"
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
1990,30,MIN,AL,146,615,551,82,164,40,3,12,80,5,4,57,73,.298,.365,.446,.811,121,246,15,3,1,3,11,*8/79D456,AS
1991,31,MIN,AL,152,661,611,92,195,29,6,15,89,11,5,31,78,.319,.352,.460,.812,119,281,27,4,8,7,4,*89,"AS,MVP-7,GG"
1992,32,MIN,AL,160,696,639,104,210,38,4,19,110,17,7,44,97,.329,.374,.490,.864,139,313,17,6,1,6,13,*8/D456,"AS,MVP-2,GG,SS"
1993,33,MIN,AL,156,682,622,89,184,39,3,22,89,8,6,47,93,.296,.349,.474,.824,120,295,15,7,1,5,7,*89D/7,AS
1994,34,MIN,AL,108,482,439,79,139,32,3,20,112,6,3,28,47,.317,.362,.540,.902,129,237,11,7,1,7,7,*9D/8,"AS,MVP-7,SS"
1995,35,MIN,AL,137,602,538,83,169,39,0,23,99,3,2,56,89,.314,.379,.515,.894,130,277,15,3,0,5,18,*9D/8456,"AS,MVP-21"
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
1990,30,MIN,AL,146,615,551,82,164,40,3,12,80,5,4,57,73,.298,.365,.446,.811,121,246,15,3,1,3,11,*8/79D456,AS
1991,31,MIN,AL,152,661,611,92,195,29,6,15,89,11,5,31,78,.319,.352,.460,.812,119,281,27,4,8,7,4,*89,"AS,MVP-7,GG"
1992,32,MIN,AL,160,696,639,104,210,38,4,19,110,17,7,44,97,.329,.374,.490,.864,139,313,17,6,1,6,13,*8/D456,"AS,MVP-2,GG,SS"
1993,33,MIN,AL,156,682,622,89,184,39,3,22,89,8,6,47,93,.296,.349,.474,.824,120,295,15,7,1,5,7,*89D/7,AS
1994,34,MIN,AL,108,482,439,79,139,32,3,20,112,6,3,28,47,.317,.362,.540,.902,129,237,11,7,1,7,7,*9D/8,"AS,MVP-7,SS"
1995,35,MIN,AL,137,602,538,83,169,39,0,23,99,3,2,56,89,.314,.379,.515,.894,130,277,15,3,0,5,18,*9D/8456,"AS,MVP-21"
12 Yrs,1783,7831,7244,1071,2304,414,57,207,1085,134,76,450,965,.318,.360,.477,.837,124,3453,188,56,23,58,85,,
Year,Age,Tm,Lg,G,PA,AB,R,H,2B,3B,HR,RBI,SB,CS,BB,SO,BA,OBP,SLG,OPS,OPS+,TB,GDP,HBP,SH,SF,IBB,Pos,Awards
1982,22,MIN-min,Rk,65,305,275,65,105,15,3,3,35,43,4,25,27,.382,.438,.491,.928,,135,,3,1,1,0,,ELZ · APPY
1983,23,MIN-min,A,138,604,548,105,172,29,7,9,97,48,11,46,62,.314,.366,.442,.808,,242,,2,3,5,2,,VIS · CALL
1984,24,MIN-min,AAA,21,87,80,9,21,2,0,1,5,8,2,4,14,.263,.294,.325,.619,,26,,0,2,1,0,,TOL · IL
1984,24,MIN,AL,128,583,557,63,165,12,5,0,31,14,7,16,69,.296,.320,.336,.655,79,187,11,4,4,2,1,*8,RoY-3
1985,25,MIN,AL,161,744,691,80,199,29,13,4,74,21,12,41,87,.288,.330,.385,.715,92,266,9,4,5,3,0,*8,MVP-21
1986,26,MIN,AL,161,723,680,119,223,37,6,31,96,20,12,34,99,.328,.366,.537,.903,142,365,14,7,2,0,4,*8,"AS,MVP-6,GG,SS"
1987,27,MIN,AL,157,668,624,96,207,32,5,28,99,12,7,32,91,.332,.367,.534,.900,132,333,16,6,0,6,7,*8/D,"AS,MVP-3,GG,SS"
1988,28,MIN,AL,158,691,657,109,234,42,5,24,121,6,7,23,83,.356,.375,.545,.920,153,358,17,2,0,9,4,*8,"AS,MVP-3,GG,SS"
1989,29,MIN,AL,159,684,635,75,215,45,4,9,85,11,4,41,59,.339,.379,.465,.843,131,295,21,3,0,5,9,*8/D,"AS,MVP-7,GG,SS"
1990,30,MIN,AL,146,615,551,82,164,40,3,12,80,5,4,57,73,.298,.365,.446,.811,121,246,15,3,1,3,11,*8/79D456,AS
1991,31,MIN,AL,152,661,611,92,195,29,6,15,89,11,5,31,78,.319,.352,.460,.812,119,281,27,4,8,7,4,*89,"AS,MVP-7,GG"
1992,32,MIN,AL,160,696,639,104,210,38,4,19,110,17,7,44,97,.329,.374,.490,.864,139,313,17,6,1,6,13,*8/D456,"AS,MVP-2,GG,SS"
1993,33,MIN,AL,156,682,622,89,184,39,3,22,89,8,6,47,93,.296,.349,.474,.824,120,295,15,7,1,5,7,*89D/7,AS
1994,34,MIN,AL,108,482,439,79,139,32,3,20,112,6,3,28,47,.317,.362,.540,.902,129,237,11,7,1,7,7,*9D/8,"AS,MVP-7,SS"
1995,35,MIN,AL,137,602,538,83,169,39,0,23,99,3,2,56,89,.314,.379,.515,.894,130,277,15,3,0,5,18,*9D/8456,"AS,MVP-21"
12 Yrs,1783,7831,7244,1071,2304,414,57,207,1085,134,76,450,965,.318,.360,.477,.837,124,3453,188,56,23,58,85,,

(1) create a list that holds all the urls you want to crawl, (2) put all your code in a function that accepts a url and csv_filename as parameters and make it crawl that url and exports to that csv_filename (3) iterate (foor loop) through the list an apply the function from step (2) — Mohamed Ali JAMAOUI, Aug 20 '17 at 20:25
I attempted to do as you suggested. It's *almost* working but the problem is in my csv output. Basically, it's appending the data as one line of the data, then two lines of the data, then three, then four, and five, and so on, until it finally appends all rows of a player's data. Then it moves to the next player and does the same thing. What I want is just all rows appended in full, and then the next player's data appended in full. — TJE, Aug 21 '17 at 13:27
There's no error message. It's working, but I guess something is wrong with how the iterator is appending the player_data to the csv in the last part of the function. I've pasted the csv output above for one player. So I have 12 seasons for this player, starting from 1982 and ending in 1995. Each row should contain data for one year, so I should have 12 rows for this one player. Instead, I get the 1982 row, then 1982 and 1983, then 1982, 1983, 1984, and so on.... until finally I get all 12 rows. — TJE, Aug 21 '17 at 13:37
@ThomasErnste it seems the lines where you append to player data and write to the file should be indented one less block. As it is, they are part of the for loop, so they get executed every loop, instead of after all the loops are done. — dogoncouch, Aug 21 '17 at 13:38
Wow, yes, that worked. Thanks! Is there a quick way for you to explain **why** that fixes the problem? — TJE, Aug 21 '17 at 13:49
@ThomasErnste With the append/write in the for loop, it was getting executed for every row. So every row, it was writing what it had so far. Now it waits until the for loop has gone over all the rows, and then writes the data. — dogoncouch, Aug 21 '17 at 13:53

How can I convert a script for collecting data from a single URL to scrape hundreds of URLs from the same domain?

0 Answers0