baseball-scraper 0.4.10

Description:

baseballscraper 0.4.10

baseball_scraper is a Python package for baseball data analysis. This package scrapes baseball-reference.com and baseballsavant.com so you don’t have to. So far, the package performs four main tasks: retrieving statcast data, pitching stats, batting stats, and division standings/team records. Data is available at the individual pitch level, as well as aggregated at the season level and over custom time periods.

Status



Statcast
data include pitch-level features such as Perceived Velocity (PV), Spin Rate (SR), Exit Velocity (EV), pitch X, Y, and Z coordinates, and more. The function `statcast(start_dt, end_dt)` pulls this data from baseballsavant.com.
Pull advanced metrics from Major League Baseball’s Statcast system. Statcast data include pitch-level features such as Perceived Velocity (PV), Spin Rate (SR), Exit Velocity (EV), pitch X, Y, and Z coordinates, and more. The function statcast(start_dt, end_dt) pulls this data from baseballsavant.com.
>>> from baseball_scraper import statcast
>>> data = statcast(start_dt='2017-06-24', end_dt='2017-06-27')
>>> data.head(2)

index pitch_type game_date release_speed release_pos_x release_pos_z
0 314 CU 2017-06-27 79.7 -1.3441 5.4075
1 332 FF 2017-06-27 98.1 -1.3547 5.4196

player_name batter pitcher events ... release_pos_y
0 Matt Bush 608070.0 456713.0 field_out ... 54.8585
1 Matt Bush 429665.0 456713.0 field_out ... 54.3470

estimated_ba_using_speedangle estimated_woba_using_speedangle woba_value
0 0.100 0.137 0.0
1 0.269 0.258 0.0

woba_denom babip_value iso_value launch_speed_angle at_bat_number pitch_number
0 1.0 0.0 0.0 3.0 64.0 1.0
1 1.0 0.0 0.0 3.0 63.0 3.0
[2 rows x 79 columns]
If start_dt and end_dt are supplied, it will return all statcast data between those two dates. If not, it will return yesterday’s data. The argument team may also be supplied with a team’s city abbreviation (i.e. BOS) to obtain only observations for games containing that team. The optional argument verbose will control whether the library updates you on its progress while it pulls the data.
For a player-specific statcast query, pull pitching or batting data using the statcast_pitcher and statcast_batter functions. These take the same start_dt and end_dt arguments as the statcast function, as well as a player_id argument. This ID comes from MLB Advanced Media, and can be obtained using the function playerid_lookup. A complete example:
>>> # Find Clayton Kershaw's player id
>>> from baseball_scraper import playerid_lookup
>>> from baseball_scraper import statcast_pitcher
>>> playerid_lookup('kershaw', 'clayton')
Gathering player lookup table. This may take a moment.

name_last name_first key_mlbam key_retro key_bbref key_fangraphs
0 kershaw clayton 477132 kersc001 kershcl01 2036

mlb_played_first mlb_played_last
0 2008.0 2017.0

>>> # His MLBAM ID is 477132, so we feed that as the player_id argument to the following function
>>> kershaw_stats = statcast_pitcher('2017-06-01', '2017-07-01', 477132)
>>> kershaw_stats.head(2)
pitch_type game_date release_speed release_pos_x release_pos_z
0 SL 2017-06-29 87.2 1.0865 6.4034
1 SL 2017-06-29 86.9 1.0195 6.4324

player_name batter pitcher events description
0 Clayton Kershaw 458913 477132 strikeout swinging_strike_blocked
1 Clayton Kershaw 458913 477132 null ball

... release_pos_y estimated_ba_using_speedangle
0 ... 54.5463 0.0
1 ... 54.7625 0.0

estimated_woba_using_speedangle woba_value woba_denom babip_value
0 0.0 0.00 1 0
1 0.0 null null null

iso_value launch_speed_angle at_bat_number pitch_number
0 0 null 57 6
1 null null 57 5

[2 rows x 78 columns]


Pitching Stats
pitching stats for players across multiple seasons, single seasons, or during a specified time period
This library contains two main functions for obtaining pitching data. For league-wide season-level pitching data, use the function pitching_stats(start_season, end_season). This will return one row per player per season, and provide all metrics made available by FanGraphs.
The second is pitching_stats_range(start_dt, end_dt). This allows you to obtain pitching data over a specific time interval, allowing you to get more granular than the FanGraphs function (for example, to see which pitcher had the strongest month of May). This query pulls data from Baseball Reference. Note that all dates should be in YYYY-MM-DD format.
If you prefer Baseball Reference to FanGraphs, there is actually a third option called pitching_stats_bref(season). This works the same as pitching_stats, but retrieves its data from Baseball Reference instead. This is typically not recommended, however, because the Baseball Reference query currently can only retrieve one season’s worth of data per request.
>>> from baseball_scraper import pitching_stats
>>> data = pitching_stats(2012, 2016)
>>> data.head()
Season Name Team Age W L ERA WAR G GS
336 2015.0 Clayton Kershaw Dodgers 27.0 16.0 7.0 2.13 8.6 33.0 33.0
236 2014.0 Clayton Kershaw Dodgers 26.0 21.0 3.0 1.77 7.6 27.0 27.0
472 2014.0 Corey Kluber Indians 28.0 18.0 9.0 2.44 7.4 34.0 34.0
235 2015.0 Jake Arrieta Cubs 29.0 22.0 6.0 1.77 7.3 33.0 33.0
256 2013.0 Clayton Kershaw Dodgers 25.0 16.0 9.0 1.83 7.1 33.0 33.0

... wSL/C (pi) wXX/C (pi) O-Swing% (pi) Z-Swing% (pi)
336 ... 1.76 22.85 0.364 0.665
236 ... 2.62 NaN 0.371 0.670
472 ... 3.92 NaN 0.336 0.598
235 ... 2.42 NaN 0.329 0.618
256 ... 0.74 NaN 0.339 0.635

Swing% (pi) O-Contact% (pi) Z-Contact% (pi) Contact% (pi) Zone% (pi)
336 0.511 0.478 0.811 0.689 0.487
236 0.525 0.536 0.831 0.730 0.515
472 0.468 0.485 0.886 0.744 0.505
235 0.468 0.595 0.856 0.762 0.483
256 0.484 0.563 0.873 0.763 0.492

Pace (pi)
336 23.4
236 23.7
472 24.6
235 23.3
256 23.4

[5 rows x 299 columns]


Batting Stats
hitting stats for players within seasons or during a specified time period
Batting stats are obtained similar to pitching stats. The function call for getting a season-level stats is batting_stats(start_season, end_season), and for a particular time range it is batting_stats_range(start_dt, end_dt). The Baseball Reference equivalent for season-level data is batting_stats_bref(season).
>>> from baseball_scraper import batting_stats_range
>>> data = batting_stats_range('2017-05-01', '2017-05-08')
>>> data.head()
Name Age #days Lev Tm G PA AB R H ... HBP
1 Jose Abreu 30 69 MLB-AL Chicago 7 31 30 5 9 ... 0
2 Lane Adams 27 69 MLB-NL Atlanta 6 6 6 0 2 ... 0
3 Matt Adams 28 68 MLB-NL St. Louis 6 9 9 2 4 ... 0
4 Jim Adduci 32 69 MLB-AL Detroit 6 24 21 3 5 ... 0
5 Tim Adleman 29 72 MLB-NL Cincinnati 1 2 2 0 0 ... 0

SH SF GDP SB CS BA OBP SLG OPS mlb_ID
1 0 0 1 0 0 0.300 0.323 0.667 0.989 547989
2 0 0 1 1 0 0.333 0.333 0.333 0.667 572669
3 0 0 0 0 0 0.444 0.444 0.778 1.222 571431
4 0 0 0 0 0 0.238 0.333 0.381 0.714 451192
5 0 0 0 0 0 0.000 0.000 0.000 0.000 534947

[5 rows x 28 columns]


Fangraphs
Various baseball projections are available at fangraphs.com. You can scrape that site using the fangraphs API. You supply it the fangraph player ID to lookup and the projection system. It will return a DataFrame with the projections.
Note, due to the use of JavaScript on that site, we use Chrome through selenium to scrape the data. Chrome must be installed on your system in order to use these APIs.
>>> from baseball_scraper import fangraphs
>>> from baseball_id import Lookup
>>> player_id = Lookup.from_names(['Khris Davis']).iloc[0].fg_id
>>> fangraphs.Scraper.instances()
['Steamer (RoS)', 'Steamer (Update)', 'ZiPS (Update)', 'Steamer600 (Update)', 'Depth Charts (RoS)', 'THE BAT (RoS)']
>>> fg = fangraphs.Scraper("Steamer (RoS)")
>>> df = fg.scrape(player_id, scrape_as=fangraphs.ScrapeType.HITTER)
>>> df.columns
Index(['index', 'Name', 'Team', 'G', 'PA', 'AB', 'H', '2B', '3B', 'HR', 'R',
'RBI', 'BB', 'SO', 'HBP', 'SB', 'CS', '-1', 'AVG', 'OBP', 'SLG', 'OPS',
'wOBA', '-1.1', 'wRC+', 'BsR', 'Fld', '-1.2', 'Off', 'Def', 'WAR',
'playerid'],
dtype='object')
>>> df
index Name Team G PA AB H ... BsR Fld -1.2 Off Def WAR playerid
60 Khris Davis Athletics 56 242 214 53 ... -0.7 -0.1 NaN 4.8 -5.9 0.7 9112

[1 rows x 32 columns]
>>> player_id = Lookup.from_names(['Max Scherzer']).iloc[0].fg_id
>>> df = fg.scrape(player_id, scrape_as=fangraphs.ScrapeType.PITCHER)
>>> df.columns
Index(['index', 'Name', 'Team', 'W', 'L', 'ERA', 'GS', 'G', 'SV', 'IP', 'H',
'ER', 'HR', 'SO', 'BB', 'WHIP', 'K/9', 'BB/9', 'FIP', 'WAR', 'RA9-WAR',
'playerid'],
dtype='object')
>>> df
index Name Team W L ERA ... K/9 BB/9 FIP WAR RA9-WAR playerid
0 5 Max Scherzer Nationals 6 3 3.04 ... 12.36 2.13 2.93 2.2 2.4 3137

[1 rows x 22 columns]


Game-by-Game Results and Schedule
The baseball_reference team scraper returns a team’s game-by-game results for a given season or date range. The resulting DataFrame includes game date, home and away teams, end result (W/L/Tie), score, winning/losing/saving pitchers, attendance, and division standing at that date.
You define the team when the scraper is created. Then can reuse the scraper to scrape specific seasons or date ranges. The team name provided is the abbreviation (i.e. NYY for New York Yankees, SEA for Seattle Mariners).
If the season argument is set to the current season, the query returns results for past games and the schedule for those that have not occurred yet.
>>> # Example: Let's take a look at the individual-game results of the 1927 Yankees
>>> from baseball_scraper import baseball_reference
>>> s = baseball_reference.TeamScraper()
>>> s.set_season(1927)
>>> data = s.scrape('NYY')
>>> data.head()
Date Tm Home_Away Opp W/L R RA Inn W-L Rank \
1 Tuesday, Apr 12 NYY Home PHA W 8.0 3.0 9.0 1-0 1.0
2 Wednesday, Apr 13 NYY Home PHA W 10.0 4.0 9.0 2-0 1.0
3 Thursday, Apr 14 NYY Home PHA T 9.0 9.0 10.0 2-0 1.0
4 Friday, Apr 15 NYY Home PHA W 6.0 3.0 9.0 3-0 1.0
5 Saturday, Apr 16 NYY Home BOS W 5.0 2.0 9.0 4-0 1.0

GB Win Loss Save Time D/N Attendance Streak
1 Tied Hoyt Grove None 2:05 D 72000.0 1
2 up 0.5 Ruether Gray None 2:15 D 8000.0 2
3 Tied None None None 2:50 D 9000.0 2
4 Tied Pennock Ehmke None 2:27 D 16000.0 3
5 up 1.0 Shocker Ruffing None 2:05 D 25000.0 4

>>> # Let get the games a team plays in a given week.
>>> import datetime as dt
>>> s.set_date_range(dt.datetime(2019,6,2), dt.datetime(2019,6,8))
>>> df = s.scrape('TOR')
>>> df.head()

Date Tm Home_Away Opp W/L R ... Save Time D/N Attendance Streak Orig. Scheduled
59 2019-06-02 TOR @ COL L 1.0 ... None 2:43 D 37861.0 -6.0 None
60 2019-06-04 TOR Home NYY W 4.0 ... Giles 3:00 N 20671.0 1.0 None
61 2019-06-05 TOR Home NYY W 11.0 ... None 3:22 N 16609.0 2.0 None
62 2019-06-06 TOR Home NYY L 2.0 ... Chapman 3:07 N 25657.0 -1.0 None
63 2019-06-07 TOR Home ARI L 2.0 ... None 2:50 N 16555.0 -2.0 None

[5 rows x 19 columns]


Team List
Using the TeamListScraper you can pull a list of active teams and a few attributes about each from baseball-reference. This has benefit mainly to get a list of abbreviations that baseball-reference uses for each team, as there doesn’t seem to be a standard. This comes in handy when wanting to use another baseball-reference scraper such as TeamList and need to input the team abbreviation.

::
>>> from baseball_scraper import baseball_reference
>>> tss = baseball_reference.TeamSummaryScraper()
>>> df = tss.scrape(2019)
>>> df.columns
Index([u'Franchise', u'Tm', u'#Bat', u'BatAge', u'R/G',
u'G', u'PA', u'AB', u'R', u'H',
u'2B', u'3B', u'HR', u'RBI', u'SB',
u'CS', u'BB', u'SO', u'BA', u'OBP',
u'SLG', u'OPS', u'OPS+', u'TB', u'GDP',
u'HBP', u'SH', u'SF', u'IBB', u'LOB'],
dtype='object')
>>> df[df.Franchise.str.endswith("Rays")].abbrev.iloc(0)[0]
'TBR'





Standings
The standings(season) function gives division standings for a given season. If the current season is chosen, it will give the most current set of standings. Otherwise, it will give the end-of-season standings for each division for the chosen season.
This function returns a list of dataframes. Each dataframe is the standings for one of MLB’s six divisions.
>>> from baseball_scraper import standings
>>> data = standings(2016)[4]
>>> print(data)
Tm W L W-L% GB
1 Chicago Cubs 103 58 .640 --
2 St. Louis Cardinals 86 76 .531 17.5
3 Pittsburgh Pirates 78 83 .484 25.0
4 Milwaukee Brewers 73 89 .451 30.5
5 Cincinnati Reds 68 94 .420 35.5


ESPN
With the ESPN scraper you can pull probable starters for a given date range. This can be useful to see the two-start pitchers for a given week and their team matchups.
>>> from baseball_scraper import espn
>>> from datetime import datetime
>>> es = espn.ProbableStartersScraper(datetime(2019,8,5), datetime(2019,8,11))
>>> df = es.scrape()
>>> df.head(10)
Date Name espn_id opponent
0 2019-08-05 Sandy Alcantara 35241 NYM
1 2019-08-05 Jacob deGrom 32796 MIA
2 2019-08-05 Jordan Lyles 31061 PIT
3 2019-08-05 Dario Agrazal 39813 MIL
4 2019-08-05 Masahiro Tanaka 33150 BAL
5 2019-08-05 Gabriel Ynoa 33651 NYY
6 2019-08-05 Lucas Giolito 32697 DET
7 2019-08-05 Spencer Turnbull 33732 CHW
8 2019-08-05 Mike Montgomery 31092 BOS
9 2019-08-05 Rick Porcello 29966 KC
>>> df[df.duplicated(['espn_id'])].Name
127 Masahiro Tanaka
128 Jacob Waguespack
130 Rick Porcello
131 Vince Velasquez
132 Jeff Samardzija
133 Mike Montgomery
134 Spencer Turnbull
135 Mike Soroka
136 Sandy Alcantara
139 Jake Odorizzi
146 Kyle Hendricks
153 Charlie Morton
155 Andrew Cashner
157 Trent Thornton
158 Jakob Junis
159 Daniel Norris
160 Jacob deGrom
161 Max Fried
162 Jordan Yamamoto
163 Jon Lester
164 Luis Castillo
165 Chris Bassitt
166 Lucas Giolito
167 Mike Minor
168 Jordan Lyles
169 Zach Plesac
170 Jose Berrios
171 Dario Agrazal
172 Michael Wacha
173 German Marquez
174 Dinelson Lamet
175 Merrill Kelly
176 Wade LeBlanc
177 Jake Arrieta
Name: Name, dtype: object
>>> df[df.Name == "Charlie Morton"]
Date Name espn_id opponent
15 2019-08-05 Charlie Morton 29155 TOR
153 2019-08-10 Charlie Morton 29155 SEA


Complete Documentation
So far this has provided a basic overview of what this package can do and how you can use it. For full documentation on available functions and their arguments, see the [docs](https://github.com/spilchen/baseball_scraper/tree/master/docs) folder.


Installation
To install baseball_scraper, simply run
pip install baseball_scraper
or, for the version currently on the repo (which may at times be more up to date):
git clone https://github.com/spilchen/baseball_scraper
cd baseball_scraper
python setup.py install


Testing
We use pytest for testing the package. It can be invoked as follows:

::
python setup.py test




Dependencies
This library depends on: Pandas, NumPy, bs4 (beautiful soup), and Requests.