[ View menu ]

November 6, 2014

When to fly to get there on time? Six million flights analyzed.

Filed in Encyclopedia ,Ideas ,R
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)

EVERY U.S. FLIGHT IN 2013 ANALYZED

Flight_Delays_By_Hour_DelayType
(click to enlarge)

If you read Decision Science News, you are probably interested in decision making, you probably fly a lot, and you probably like making decisions about flying.

Data of the type the U.S. Government provides enable us to predict how delayed we will be when we fly at various hours of the day.

To make the plot above, we analyzed every single flight in the United States in 2013 for which there were Bureau of Transportation Statistics data. Filtering out flights between midnight and 6AM that leaves us with a little over six million flights (6,283,085 flights, to be precise). The BTS defines delay as the difference between the time the plane actually arrived and the time listed in the computerized reservation system. Many flights got in early, but because we’re just interested in delays (not speedups), we negative delays with zeroes.

What do we learn?

The later you leave, the greater the average delay you will face until around 6PM when things flatten out and 10PM when we see benefits in leaving later. It makes sense that delays increase as the day goes on because, we understand, the primary cause of delays is waiting for the plane to arrive from another city. The first flights out in the morning don’t have this problem.

About 60% of flights had no delay at all (3,726,061/6,283,085 or 59.3% to be precise). This has something to do with padding the expected arrival times in the computerized reservation system. Hence all the “negative” delays.

Leaving at 11PM gives you the same delay as leaving at 11AM. Miracle of miracles. Want a rule of thumb? Try not to leave between 11AM and 11PM.

The arrival and departure curves are quite similar. To save space, we’ll only look at departure delays from here on.

Now, you may be thinking “20 minutes delay if you depart at the worst possible time? That’s not such a big deal.” But remember, these are averages and 60% of the time there will be zero delay. To show you how bad things can get, here we plot the 95th and 75th percentiles of the delay distribution:

Flight_Delays_By_Hour_95thIf you leave at the worst time of day,  1 time in 4 you’ll be delayed more than 20 minutes, and 1 time in 20  you’ll be delayed more than an hour and a half!

Do different airports have differing delay patterns? One might expect them to due to weather, total number of flights, longitude and the like. We isolate the ten airports with the most passenger traffic below:

Flight_Delays_By_Hour_Airport_Top5

Flight_Delays_By_Hour_Airport_6to10

In an early analysis, we thought we’d discovered something pretty cool about day of the week effects. We had chosen two months at random and noticed certain days were predictably worse than others. But then, when we looked at two different months, different days emerged as the worst ones. Digging deeper, we found that the day-0f-week effects are attributable mostly to rather random events which change from month to month. Here we look at median (not mean) delays on every day of 2013. Each panel represents one month.

Flight_Delays_By_Departure_Date

The big spike on April 18, 2013? Five inches of rain in Chicago. December 9th, 2013? Delays are mostly due to winter weather in Texas. These little bumps can really alter the day-of-week findings.

Bon voyage!

R-code, as usual, for those who want it. To get the flight data, just go to … aw heck, I’ll be nice and let you download my cleaned up copy (25 Mb)

This is our first use of Hadley Wickham’s tidyr package. We like it!

ADDENDUM

1. We just learned of some extensive analyses pre-2009 flight data you might find interesting. See the FlowingData blog post. The supplemental information in this paper has some interesting analysis of flight delays. For example, hub airports tend to have a lot of outbound delays because they hold planes when an incoming flight is late. This leads to a lot of arrival delays at non-hub airports. See wicklin-supplemental.pdf page 7.

2. Poking around at this link, we were above to find somewhat steady day of week patterns in this poster which draws on multi-year data.

July 13, 2015

Our travel tips

Filed in Ideas
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)

A PACKING LIST AND MORE

jl

Esquire recently posted some of Anthony Bourdain’s travel tips. Some good stuff there. Seeing the list reminded the editorial team at Decision Science News that we’ve always wanted to do a post on this. We travel a lot.

A PACKING LIST SAVES US TIME AND PREVENTS ERRORS

This packing list is based on one that David Allen (of Getting Things Done fame) shared. We’ve modified it to suit our needs. We pack from a list every trip. It gets packing time down to 20 minutes and zero errors. Here is the PDF (source for editing), ready to print four-per-page.

=ESSENTIALS
Socks
Underwear
T-shirts short + long sleeve
Shirts
Pants
Shoes
Belt
Glasses + contacts
Razor + cord
=FITNESS
Sneakers
Exercise clothes
Bathing suit
=BUSINESS ONLY
Dress shirts
Dress pants
Suit or jacket
Tie
Cufflinks #so easy to forget
=WEATHER DEPENDING
Sunglasses
Sandals
Shorts
Sunscreen
Umbrella
Gloves
Coat
Sweater
=IN WALLET
Business cards
Airport lounge ID card
=IN WALLET-INTERNATIONAL
Foreign credit cards
Foreign transit cards (UK: Oyster; NL: OV Chip)
=IN CARRY ON
Toiletries ziploc
Phone + headphones w/ microphone
Hoodie #avoid freezing on the plane
Inflatable Neck pillow #see below
Laptop + charger
USB wall plug adapter
Retractable micro USB cord #see below
Plane/train/car/hotel info
Maps (cached to phone) + directions
USB memory sticks
Conference programme
Prescription medicine
=IN CARRY ON-INTERNATIONAL
Passport
Foreign currency
Foreign plug adapters #see below
=DO BEFORE LEAVING
Weather checked?
Computer synced?
Presentation to USB and emailed to self?
Withdrew cash?
Checked in + chosen seats?

GETTING A PHONE WITH FREE GLOBAL DATA WAS GAME CHANGING

Since we moved from Verizon to t-mobile, our phone now works everywhere in the world and gets free data everywhere in the world. Much better than schlepping around a second unlocked phone and switching sim cards.

INFLATABLE NECK PILLOWS MAKE US FEEL GOOD

For a long time, we looked at those silly looking people with their neck pillows and felt superior. Then we tried one. They really make you feel better. We’ve tried four kinds, including some expensive ones, but our favorite is the Samsonite Inflatable Neck Pillow.  It folds up to an impossibly small size and disappears so you won’t look a doofus marching around the airport with one tied to your bag.

IT’S NOT THE JET LAG, IT’S THE SLEEP DEPRIVATION

More information.

BOARDING TIME CAN BE UP TO 10 MINUTES BEFORE THE PUBLISHED BOARDING TIME

This matters if you have status with the airline and get to board first. If you show up at the time on the ticket, you may be boarding with group 4.

WE PREVENT LIQUIDS FROM LEAKING THIS WAY

Two things. 1) Squeeze a little air out 2) Put a little square of plastic wrap over the mouth of the bottle. Screw the cap on. Voila.

WE LIKE RETRACTABLE USB CORDS

Like these. Keeps things tidy and untangled. Apologize to the squirrels who were nesting in your bag.

WE LIKE INTERNATIONAL PLUG ADAPTERS WITH BUILT-IN USB PORTS

Like these. Use your computer and charge your phone at the same time. Amaze your friends with what you can charge in one outlet, anywhere in the world.

WE LIKE COMPRESSION BAGS

Like these. They make your stuff, especially winter stuff, take up less space. And they act as laundry bags for the return trip. And they will keep a wet bathing suit from getting the rest of your stuff wet. And you don’t need a vacuum cleaner to use them. Just seal and roll. The air escapes through magical one-way valves. Not recommended for things that wrinkle easily. BTW, dry cleaning bags do a pretty good job of keeping things wrinkle free.

MEMORIZING OUR PASSPORT NUMBER WAS WORTH IT

If you travel internationally a lot, this is more useful than it sounds. You get asked for it surprisingly often, and you never feel like digging up your passport when filling out forms. We even have advice on how to remember long numbers. Lock it in once and you’re good for life.

SWIM DON’T RUN

And avoid packing an extra pair of shoes for exercise. We read about this in an interview once.

WE TRY NOT TO BOOK FLIGHTS THAT DEPART BETWEEN 11AM and 10PM

This helps us avoid flight delays.

 WE ALWAYS RETURN HOME WITH ZERO FOREIGN COINS

And this is how we do it.

 

If you have tips, feel free to share them in the comments.

Photo credit: http://www.flickr.com/photos/araswami/1948449158/

December 15, 2014

Decide which frequent flyer program is best for your city

Filed in Programs ,R
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)

INTERACTIVE TOOL TO SEE THE NUMBER OF DEPARTURES

dc
Click to access the tool

We’ve posted in the past about which airline you should be loyal to, but we always felt guilty because we only showed results for the New York metro area and there are Decision Science News readers all over the USA. (Actually, there are DSN readers all over the World too, though in most countries it’s an easy decision: go with the national airline).

Since then, we’ve learned about data for every flight in the USA that makes it rather straightforward to generate for every US metro area and airline, the number of departures.

Try out the new tool here: Decide which frequent flyer program to be loyal to.

Just type in the name of your metro area in the search box and you’ll see the number of departures by airline for your metro area only. The data comprise every flight taken in the USA in 2013.

There will be airlines you don’t recognize (like ExpressJet, etc.). They are just regional carriers that fly for the big airlines. To find out who they’re flying for, just look them up in the Wikipedia. Add things together to get the totals for the big frequent flyer programs. We’d do it for you, but, well, we’ve spent long enough on this.

This tool was made with the DataTables plugin for jQuery, R, and Hadley Wickham’s dplyr powertool.

Here’s the code. You can use my pre-made 2013 flights data file.

Get the L_CARRIER_HISTORY file from the US government.

Get the L_CITY_MARKET_ID file from the US government, too.

H/T To the mighty Jake Hofman for thinking with me about how to think about this.

CODE

August 24, 2009

Transition probabilities

Filed in Conferences ,Ideas ,Tools
Subscribe to Decision Science News by Email (one email per week, easy unsubscribe)

PROBABILISTIC INFORMATION ON WHETHER YOUR FLIGHT WILL BE LATE

fc

DSN reader Yael sends along this NY Times InTransit piece on Flight Caster, a web site that uses historical data to generate probabilities that flights will be late. DSN likes the following things:

  • The idea of FlightCaster
  • The idea of ubiquitous probabilistic information in the age of the networked database and smartphone
  • The idea that accurate probabilistic information in the environment will encourage the public to embrace and understand  probabilities

We thought Flight Caster would be useful this week, as Decision Science News is travelling to the SPUDM: Subjective Probability, Utility, and Decision Making Conference in Italy, but unfortunately Flight Caster does not do flights outside the US.

However, it does remind the DSN Editor of the time in the 1990s when, just before a trip to Milan, he asked an Italian colleague what percentage of Alitalia flights were late. “What percentage?” The colleague blinked and leaned forward to make sure he didn’t misunderstand. “Hundred!” he laughed, looking like he’d just been asked to add ninety-nine and one.