The Problem of Optimization

A Love Letter to Convex Optimization

Optimization is one of the mathematical concepts that most closely mirrors city life. It’s the process of making the best possible choice within limits.

For a city, that could mean minimizing utility shutoffs, maximizing bus ridership, or balancing housing density with green space. The city’s “objective” is the outcome we care about—reliable transit, affordable housing, safe neighborhoods. But the knobs we can actually turn—staffing, zoning, program budgets—are limited. Optimization is about tuning those knobs carefully to get the best results.

Think of it like adjusting a thermostat: you can’t change the weather outside, but you can tweak the dials to keep the house livable. Cities work the same way.

---

Problem Setting: the Best Option Under Constraints

City problems are complex. A bus delay might depend on operator staffing, traffic congestion, and even weather. A housing plan might involve land use, building codes, and neighborhood feedback. You can’t test every possible combination—it’s just too many.

That’s where constraints make the problem manageable.

• Transit planners can’t assume infinite buses; they optimize routes given the current fleet.
• Community centers can’t be on every corner; they maximize reach given a budget for just a handful of new sites.

The math captures this as:

$$\min_{x \in \mathcal{X}} f(x) \quad \text{subject to} \quad g_i(x) \leq 0, \, h_j(x) = 0$$

Which reads: find the best option $x$ that minimizes the cost or maximizes the benefit $f(x)$, while still obeying the rules $g_i$ and $h_j$.

• $f(x)$: the goal (e.g., average bus delay).
• $g_i(x) \leq 0$: inequality constraints (e.g., “drivers must not work more than 8 hours”).
• $h_j(x) = 0$: equality constraints (e.g., “every bus route must cover exactly 20 stops”).
• $\mathcal{X}$: the feasible set (the space of choices the city can realistically make).

In other words: make the best choice that still respects reality.

---

Why Convexity Matters

Optimization is tough, but convex optimization is special.

In convex problems, the “valleys” of the problem landscape are shaped in a way that guarantees any local improvement leads to the best overall solution.

This is powerful in cities:

• If you’re minimizing bus delays, a convex problem ensures that once you find an efficient schedule, you know it’s the best possible one—not just “good enough.”
• If you’re optimizing affordable housing placement, convexity ensures that local tweaks (like shifting one site) can only improve things, never trap you in a dead end.

Convexity gives urban planners mathematical certainty in a world where certainty is rare.

---

Important Formulas and City Methods

Gradient Descent

$$x_{t+1} = x_t - \eta \nabla f(x_t)$$

This is the “trial-and-error” method. Start with a guess, then take small steps downhill along the slope until you reach the valley.

• In transit, this could mean iteratively testing different driver shift times and trimming delays each round.
• In housing, it could be adjusting rent subsidy levels step by step until evictions stabilize.

It’s incremental improvement—city government style.

---

Lagrangian Duality

$$\mathcal{L}(x, \lambda, \nu) = f(x) + \sum_i \lambda_i g_i(x) + \sum_j \nu_j h_j(x)$$

Cities are full of rules. You can’t just bulldoze your way to efficiency—you have to respect labor laws, safety codes, and equity mandates. The Lagrangian method adds these rules directly into the optimization formula.

• The $\lambda_i$ multipliers represent how much the constraints “cost” when they bind (e.g., a hard budget cap).
• The $\nu_j$ terms enforce strict requirements (e.g., each neighborhood must have a fire station).

This is like doing city planning with a lawyer looking over your shoulder—every choice is balanced against regulation.

---

Newton’s Method

$$x_{t+1} = x_t - [\nabla^2 f(x_t)]^{-1} \nabla f(x_t)$$

Newton’s method takes into account not just the slope, but also the curvature of the problem. It’s like having a map of the valley instead of just walking downhill blind.

• For traffic optimization, this means adjusting not just today’s delays, but how small changes ripple through the entire network tomorrow.
• For budgeting, it’s the difference between cutting line items one by one versus anticipating how those cuts reshape the whole fiscal landscape.

It’s more work per step, but it gets you to the best solution faster.

---

Bringing It All Together

Convex optimization is the quiet art behind many city decisions.

It combines:

• Realism (constraints reflect budgets, zoning, and capacity).
• Certainty (convexity ensures the solution is globally optimal).
• Efficiency (methods like gradient descent and Newton’s method make the search tractable).

The math may look abstract, but this is the city adapting to its own complexity: buses arriving on time, neighborhoods designed for livability, programs serving those who need them most.

Optimization is not about perfection—it’s about making the best possible choice under the limits of reality. And in that sense, cities themselves are living proofs of convex optimization in action.