lec08

Lecture 8. Duality of Linear Programming

8.1 LP duality

Consider the following LP: One can solve this by geometric argument or simplex method. The optimal solution is and the optimal value is . Can we prove it by some simple observations from the linear program itself?
Clearly since and . But we can do better actually. It is also easy to see that since In general, given , , and the LP constraints , we can assign each constraint a coefficient as follows: If and for all , then is an upper bound, which further implies that We want the upper bound as small as possible. Therefore, we would like to solve the following LP: which is called the dual linear program. Correspondingly, we call () the primal.

Proposition

The dual of the dual is the primal.

Proof

The dual program can be rewritten as: It is clear that the dual of dual is: which is equivalent to the primal.

We note that the primal problem may have different forms. If the primal has a constraint , then we have the corresponding variable in the dual. If the primal has a constraint , then we have the corresponding variable unconstrained in sign in the dual.

Now we use the form , subject to , . By the discussion above, we know that the optimal value of the dual problem gives an upper bound of the primal problem. This property is called the weak duality.

Theorem (Weak duality theorem)

If is feasible for primal and is feasible for dual, then .

Proof

Since and , we have .

Corollary

If is unbounded above, then the dual problem is infeasible.

However, if the primal problem is infeasible, we cannot conclude that the dual problem is unbounded below. It is possible that both primal and dual problems are infeasible.

Example

Consider the following problem: Its dual problem is It is easy to check that both of them are infeasible.

We’ve already known that any feasible solution of the dual gives an upper to the primal. The question is that, is there any gap between the optimal value of the primal and the optimal value of the dual?

Theorem (Strong duality theorem)

If the primal problem has a finite optimal solution , then the dual problem also has a finite optimal solution with the same optimal value of the primal. Namely, it always holds that .

Now we can complete the following table.

	unbounded	infeasible	optimal
unbounded	×	√	×
infeasible	√	√	×
optimal	×	×	√

The proof of the strong duality is an application of the Farkas’ lemma, which we have introduced in Lecture 4.

Theorem (Farkas' lemma)

Let . Then exactly one of the following sets is empty:

;
.

To apply this lemma, we consider the following corollary.

Corollary

Suppose and . Then exactly one of the followings is true:

There exists such that and .
There exists such that , , and .

Let be the cone of column vectors of . The corollary tells us that either intersects the region , or there exists a hyperplane strictly separating and , where and .

Proof of the corollary

Let . Applying the Farkas' lemma to and , it gives that exactly one of the followings is true:

There exists such that ;
There exists such that and .

Note that item 1 is equivalent to such that , and item 2 is equivalent to , , and . So we are done.

We are now ready to prove the strong duality theorem.

Proof of the strong duality theorem

Without loss of generality, we assume that the dual problem has an optimal solution (instead of the primal problem in the statement). Suppose the strong duality is not true. Then there does not exist feasible solution of the primal such that .
Let . It is equivalent to that there does not exists such that Then the corollary of the Farkas' Lemma shows that there exists and such that Now we can claim that is not an optimal solution to the dual problem, which contradicts to our assumption. Consider the following two cases.

Case 1. . Then we have and . Thus, and . Noting that is also a feasible solution to the dual problem. But .
Case 2. . Dividing on both sides, it leads to which implies that and . So is a feasible solution to the dual problem, and better than .

The proof is concluded by the contradictions in both cases.

In the proof of the corollary of the Farkas’ lemma, we employ the matrix , which looks similar to the standard form of the linear program. In fact, if we consider the standard form, the proof of the strong duality can apply the Farkas’ lemma directly, instead of using the corollary.

Complementary slackness

As an application of the strong duality of linear program, the following theorem reveals some relations between the optimal solutions to the primal and the dual.

Theorem (Complementary slackness)

Suppose and are feasible solutions to the primal problem and the dual problem, respectively. Then and are optimal solutions if and only if

The complementary slackness shows that if the -th constraint of the primal at the optimal solution is not tight, then the corresponding variable is in the optimal solution of the dual, and vice versa. Namely, we have that

Proof

It is clear that . Then and are optimal solutions if and only if , that is,

8.2 Applications of linear programming duality

We now consider some applications of the strong duality theorem.

The first example is the zero-sum games. We can show that the strong duality theorem can be used to prove minimax theorem, a central theorem in game theory that is even considered as the starting point of game theory. Consider the rock-scissors-paper game between two players, the row player and the column player. Each player can play one of the three moves: R(rock), S(scissors), or P(paper). Two players play simultaneously. The payoff matrix of this game is given in , where each entry corresponds to the gain for the row player, which is also the loss of the column player.

G	R	S	P
R	0	1	-1
S	-1	0	1
P	1	-1	0

For example, if the row player plays R and the column player plays S, the row player receives payoff 1, and the column player receives payoff −1 (the column player’s loss is 1). In this game, the sum of all the players payoffs always equals to 0. Games satisfy this property are called zero-sum games.

A strategy of a player is a probability distribution over {R,S,P}. If the strategy specifies that one of the three moves is played with probability 1, this strategy is called a pure strategy. Otherwise, it is a mixed strategy. Fix one player’s strategy, the best response for the other player is a strategy that maximizes this player’s payoff. For example, if the column player plays the strategy (i.e., the column player plays R), the best response for the row player is (i.e., the row player plays P), which gives the row player payoff 1. If the column player plays , the best response for the row player is again . In general, there may be more than one best response, and there is always a best response that only uses pure strategy. For example, if the column player plays , then any strategy of the row player is a best response, which always gives the row player payoff 0 (check this!)

In general, suppose each of the two players can play one of the moves. Let be the strategy of the row player, be the strategy of the column player, and be the payoff matrix, where is the payoff for the row player if the row player plays and the column player plays . The expected payoff for the row player is We now assume both the two players are smart enough, and both of them would like to maximize their own benefits.

Given a fixed strategy of the row player, the column player’s best strategy is to minimize . So the goal of the row player is to choose to maximize , that is, .
Given a fixed strategy of the column player, the row player’s best strategy is to maximize . So the goal of the column player is to choose to minimize , that is, .

We claim that their goals can be reformulated as the primal and dual linear programs. To this end, we now consider a simple example.
For instance, let .

If row choose strategy , then the payoff of column is . The column player will choose a strategy so that the payoff is as large as possible. So the payoff of column is .
Hence the goal of row is: , which can be reformulated as
If column choose strategy , then the payoff of row is . The row player will choose a strategy so that the payoff is as large as possible. So the payoff of row is .
Hence the goal of column is: , which can be reformulated as One can see that problem if the dual of . Hence the equality follows from the strong duality theorem. This is formally characterized by the following minimax theorem.

Theorem (von Neumann's Minimax Theorem)

Given a zero-sum game with payoff matrix , we have

Another example of duality is the maximum matching and minimum vertex cover problem in graph theory. Given an undirected graph , a matching is a set of edges such that no edge in the set is a loop and no two edges share common vertices, and a vertex cover is a set of vertices that includes at least one endpoint of every edge. Here are some examples of matchings and vertex covers.
Pasted image 20231022142329.png

Pasted image 20231022142348.png

Clearly the set is a matching and is a vertex cover. So we consider the problem of maximum matching and minimum vertex cover.

Theorem (Kőnig's theorem)

Suppose the graph is a bipartite graph, namely, for some disjoint and , such that . Then the size of its maximum matching equals the size of its minimum vertex cover.

We can write an integer programming formulation of the maximum matching problem. Any matching can be represented by variables such that if the edge and otherwise. Conversely, any assignment to can represent a matching if and for all . So the problem can be formulated as If we relax the constraints to be , it can be reformulated as a linear program The relaxed problem is called the maximum fractional matching problem.

Analogously, for each vertex we can assign a variable to represent whether is in the vertex cover. We relax the constraints again. Then we obtain the problem of maximum fractional vertex cover as follows It is easy to verify that the fractional minimum vertex cover problem is the dual of the fractional maximum matching problem. So we know that for any graph, the size of the maximum fractional matching equals the size of the minimum fractional vertex cover.

In a bipartite graph , we can further show that the size of the maximum fractional matching equals the size of the maximum matching. Given a fractional matching , consider the subgraph consisting of fractional edges where .

Case 1. There exists a cycle . Note that is an even number since is bipartite. Let . Then add to for all odd , and subtract from for all even (we assume that ). The resulting satisfy all constraints and the size of the fractional matching remains the same.
Case 2. There is no cycles. Then choose any path . Note that all edges incident to has except . So if belongs to but . Similarly, if belongs to but . Again, let . Then subtract from for all even , and add to for all odd (we assume that ). Now the resulting satisfy all constraints and the size of the fractional matching is nondecreasing.
Each operation decreases the number of fractional edges. So there is no fractional edges after finite many operations. Consequently, any fractional matching can be converted into an integral matching with equal or greater size, which implies that the size of the maximum fractional matching equals the size of the maximum matching.

We can also show that the size of the minimum fractional vertex cover equals the size of the minimum vertex cover in a bipartite graph .
We construct a vertex cover as follows. Pick a real number uniformly at random. For every , let if , and for every , let if . Note that for every and , if , then . So it is easy to see that at least one of is in , which asserts that is a vertex cover.
Next, we calculate the expected size of . Clearly, for any , . By the linearity of expectation, , which is exactly the size of the minimum fractional vertex cover. Thus, there exists such that the vertex cover constructed by has the size .

Overall, we conclude that in any bipartite graph, the size of the maximum matching equals the size of the minimum vertex cover.