lec07

Suppose a factory can produce two kinds of products. The profit for producing one unit of the first kind of products is 1, and the profit for producing one unit of the second kind of products is 6. Suppose a factory has a limited amount of resources such that

Given these, you are to decide how many units of the first kind of products and how many units of the second kind of products to produce respectively, so that the overall profit is maximized. This problem can be formulated as follows:
This is a typical linear program (LP). In a linear program, we have a linear objective that needs to be maximized (or minimized), and we have a bundle of linear constraints.

A linear program may have both equality and inequality constraints. It is easy to rewrite equality constraints by inequalities. Can we rewrite inequality constraints by equalities? In fact, the standard form of linear programming only allows equality constraints and a special type of inequalities.

Now the question is how to convert a linear program into the standard form? Clearly the third requirement is trivial. We only need to consider the first two requirement.
First, for the second requirement, consider a simple example . We can add a slack variable : together with . Note that it is equivalent to the original inequality. Suppose we have another example . Then multiply the inequality by and add a slack variable .
Next we consider the first requirement. If a variable has non-positive constraints, such as , we can easily let and substitute for in the linear program. If some variable, for example , is unconstrained in sign, we can replace it by and require and to be non-negative.

The standard form is useful in algorithm design and analysis, especially in the algorithms based on primal dual method.

For convenience, in this course we will use the form Note that the feasible set is a polyhedron since it is the intersection of some halfspaces.

When a linear program achieve its optimal value , intuitively the hyperplane passes through a vertex of the feasible set. We may use this observation to solve linear programs geometrically.

First, we define what a vertex and an extreme point are.

An important fact is that, these two types of points are equivalent for polyhedra.

However, a polyhedron may not contain any vertex.

We now prove the fundamental theorem of linear programming.
Let be the feasible set of a linear programming and be the set of optimal solution. Assume . Since , there is no line in and thus no line in . Note that is also a polyhedron since where is the optimal value of the objective function. By the above proposition, we know that has an extreme point . Now it suffices to show that is also an extreme point in .
Suppose is not an extreme point in , then there exists and such that . We have since is an optimal solution. Thus, . It implies that , which leads to a contradiction.

The fundamental theorem of linear programming gives us an algorithm to solve linear programs by enumerating all () vertices of the feasible set.

However, consider the -dimensional cube . Only constraints produce a polyhedron of vertices.

We now introduce a (usually) efficient algorithm: the simplex method. We remark here that the simplex method is not a polynomial-time algorithm. However, it runs fast except for some artificially designed cases.

The key idea is that when we find a vertex of the feasible set, move from the current vertex to a "better" neighbor, where two vertices are neighbors if they share tight constraints.
Assume is a feasible set. Then we can start from the origin point . It is clear that there are neighbors of the origin, and each of them has zero coordinates.
How can we know whether a neighbor is "better"? Note that our goal is to compute . If , the objective function is "better" as long as . To this end, we can choose such that and increase to until some constraint is tight. Now there are constraints tight: Then we shift coordinates so that becomes the origin point. It suffices to let , , .

As shown above, the first step is to choose a variable whose coefficient is positive, and then increase it. What should we do if we have multiple variables who have negative coefficients?

The following example shows we may encounter some tricky problems if we choose a wrong variable. Consider the linear program with the same objective function as above, but the constraints are

Clearly it is possible to fail in this case, which is called degeneracy. Moreover, since we have exponentially many vertices for a polytope in , this algorithm may have exponential running time if we do not have a clever choice at each step.

In fact, it is a central open problem if there is an implementation of the simplex method that always runs in polynomial time. We may have some natural choices when deciding the next vertex to move to in the simplex method. For example, we can choose an adjacent vertex with the highest cost, or, we can choose an adjacent vertex randomly. However, for each of those implementations of the simplex method that researchers have thought about, there are examples that makes the implementation runs in exponential time.

Fortunately, those examples are rare: in practice, the simplex method is very efficient. Theoretical computer scientists have been spending efforts on explaining this phenomenon. There is an entire field called smoothed analysis, introduced by Shang-hua Teng & Daniel Spielman, that deals with this. Most notably, Teng & Spielman showed that, if we add a random Gaussian noise to either or in , that is, for example, let where is an i.i.d.(independent and identically distributed) Gaussian random variable, simplex method runs in polynomial time in the average case.

We also remark that, although the simplex method does not provably run in polynomial time (yet), there exist other methods for solving linear programs that provably run in polynomial time. We will discuss this more later in this course. Therefore, whenever a problem can be formulated by a linear program, it is polynomial time solvable.

Now the question is, what if the origin point is not feasible? If there exists a known feasible solution such that , then let and further let to guarantee that all variables are nonnegative.

However, what if there is no known feasible solutions? The following two-phase simple method gives an algorithm to find a feasible solution of a linear program.