Life lessons from an optimizer
My nerdy thought process origin story
Last week, I wrote about my new mental model of work. A few readers commented on my analytical approach to the topic. This led me to reflect on the origins of my decision-making process.
I worked with optimization software for the majority of my 20-year airline career. I didn’t realize how much it shaped my thinking until I started writing about it.
Crew pairing optimization, or building flight schedules into crew schedules, is among the world’s most difficult math problems. It’s the number one way an airline can influence crew cost, staffing, and operational performance.
Every flight must be staffed with the required number of pilots and flight attendants. Schedules must comply with union and government regulations. An optimal solution operates well, provides quality of life, and minimizes costs.
My previous airline had the world's largest such problem.
Each month, crew planners solved a ~120,000-piece puzzle with billions of possible solutions. Using principles of the scientific method, we’d adjust one of ~200 parameters to different degrees, run scenarios, analyze the outputs, and repeat. Eventually we selected a solution to publish with the monthly crew bids.
We joked that we never really finished the schedule, we just ran out of time.
Skillful use of optimization software can be applied to other parts of life.
Life Lesson #1: Use rules sparingly
Using the Waze app for navigation, I see an odd routing taking 10-15 minutes longer than expected. I realized in my settings “Avoid toll roads” is switched on. Because this is a hard yes/no rule, Waze won't show me the faster toll road option.
A more flexible option would say, “Avoid toll roads unless it saves me ___ minutes”.
Crew pairing optimizers behave similarly when given rules. A typical pairing optimizer has the following inputs:
Flight schedule
Rules: tells the optimizer what is and isn’t allowed
Costs: values/scores guiding solutions to the desired result
Rules are useful to:
Ensure schedules built are within government and union regulations (crew duty time, minimum rest, etc.)
Limit possible solutions for a large problem, increasing speed
Sometimes rules have unintended consequences, though.
A solution would come back with an uncovered flight. Sometimes it was easy for our human eyes to spot a much “better” solution. Sometimes we’d just scratch our heads wondering why a solution looked so bad.
Diagnosing a rule problem took time we didn’t have when facing deadlines.
Often the issue was a rule that should have a penalty or "cost" instead.
Working with the optimizer taught me to:
Use rules sparingly, revisit often, and consider when a rule should instead be a "cost".
This also reminds me of a day I told my boss I would never date someone I knew from work. We were discussing someone we knew who was in a relationship with a coworker, and how unnecessarily complicated it was. Later that night, at a coworker’s going away party, I met a fellow employee who had just moved to Dallas. This person is now my husband.
I don’t like rules.
Life Lesson #2: Understand trade-offs
Our attention, time, and money are finite. Is cutting my vacation short a day worth saving $100 on airfare? Is driving an extra two hours to hike a nice trail better than running in my neighborhood and having time to read a book? Is taking a year off to learn worth losing a year of income?
In crew pairing optimization, planners must be crystal clear on priorities.
Crew pairing optimization objective: assign all flights into crew schedules with the lowest cost.
The term “cost” led to misconceptions.
In this case, cost did not equal dollars but an overall sum of penalties.
The optimizer had over 100 different “cost” parameters driving solution results. Crew planners assigned numerical values to each parameter. Things to avoid had penalties, things to achieve had discounts.
Crew pay hours and hotel costs had direct dollar equivalents. Parameters driving operational efficiency and quality of life did not. All had a place and weight in the solution.
Assigning numbers to desired outcomes
➡️ 10 cost is a tie-breaker
When solving a problem with billions of possible solutions, small costs mattered, but not at the expense of more important outcomes. For example, giving Seattle overnights to Oakland-based crews (many lived in Seattle and commuted) would get a bonus, but a relatively small one. It’s a “nice-to-have”—not worth creating undesirable extra work days, crew aircraft swaps, or short overnights.
➡️ 10,000 cost is nearly a rule
The optimizer would take on all sorts of other costs to avoid it. Doing 9 things we didn’t like at 1,000 apiece < 10,000 penalty for one instance. Giving too many parameters high penalties resulted in junk solutions.
Sometimes we assigned values based on detailed analysis. More often we used intuition and experimentation.
The woman who trained me said using the optimizer was half math, half art.
Working with the optimizer taught me:
All decisions have trade-offs; be clear about priorities to get desired results
Life Lesson #3: Communicate what success looks like
"These schedules suck"--Crew Planner
Solving a puzzle with billions of possible solutions often felt like playing "whack-a-mole". We'd run a scenario, get something we didn't like, tweak a parameter, run another one, and something else we hated would pop out.
Our operations research (OR) developers who supported the optimizer were our heroes.
"What don't you like about the solution?"--OR Hero
We had to be specific.
"It built three Hawaii trips from LAX the first Tuesday, and only one the next Tuesday. I can't build a monthly schedule out of that. Our staffing will be out of balance unless we have the same number each Tuesday.
"The optimizer doesn't know you want that, but it seems pretty easy to add."
Within a week, they'd give us a new parameter to test and a way to measure the output on a report.
People aren't that different.
Too often we speak in generalities. We can't get what we want without being specific about the desired outcome. It also helps to explain why.
Well, the optimizer doesn't care why. But sometimes explaining "why" would help the the OR Hero think of an even better way to reach that outcome.
Working with the optimizer taught me:
Clear communication = better chance of having needs met
I say “life lessons from an optimizer” in jest. However, I am grateful for the skills I gained using optimization software:
Use rules sparingly
Understand tradeoffs; be clear about priorities
Clearly communicate desired results
