Lecture 3. Convex Sets
...


3.1 Affine sets
...

Affine sets are generalization of lines. Given two points , the line passing through can be represented by Note that . So we have the following definition of affine combination.

Definition (Affine combination)

Given , is an affine combination of if .

A set is affine if it is closed under affine combinations.

Definition (Affine set)

A set is an affine set, if for all , for all points , any affine combination of is still in .

Example
  • A line is an affine set;
  • is an affine set;
  • Given and , the hyperplane is an affine set;
  • In general, given and , the solution set of the system of linear equations is an affine set.

Note that if , the solution set is a hyperplane. If and , is the intersection of hyperplanes.

Proof

Given , we have . So for any , .

Why can we only verify affine combinations of two points in ? Suppose we have an affine combination for points . Since , clearly there must exists two of them such that their sum is non-zero. Assume that . Then we have If any affine combination of two points is still in , then is in and thus is in . For an affine combination of more than points, we can rewrite it in a similar way recursively. So it suffices to verify affine combinations of points.

We have shown that the solution to each linear equation is an affine set. Conversely, any affine set is also a solution set to a system of linear equations.

Proposition

Any affine set is the solution set to a system of linear equations.

Proof

If is an affine set, pick an arbitrary point . Then we claim that the following set is a linear space. For all , we have by definition. Hence, for any , Therefore, .
Since can be represented as , then can be represented as , which is the solution set to .

Roughly speaking, affine can be viewed as linear added by some bias term. Similar to the linear map, we can define an affine map by for some and . We can also define affinely independent points as follows.

Definition (Affine independence)

Given points , we say they are affinely independent, if there does not exist such that , and Equivalently, are affinely independent, if and only if are linearly independent.

Clearly, there are at most affinely independent points in , since there are at most linearly independent vectors in .

3.2 Convex sets
...

Similar to the definition of lines, we can define the segment from to by Note that the difference between lines and segments is the range of . Again, since , we have the following definition.

Definition (Convex combination)

Given , is a convex combination of if and for all , .

A set is convex if it is closed under convex combinations.

Definition (Convex set)

A set is a convex set, if for all , for all points , any convex combination of is still in .

In particular, we can define the convex hull of any set.

Definition (Convex hull)

The convex hull of a set is the set of all convex combinations of points in , namely,

Clearly, for any set , its convex hull is a convex set.

For a general set , if we would like to show that is convex, using the same argument we used in the section of affine sets, we only need to show that any convex combination of two arbitrary points in is still in .

Question

If we would like to determine the convex hull of some set , can we only check convex combinations of any two points? If not, how many points are sufficient?

Tip

At most points in are sufficient. Because points are affinely dependent, there exists such that , and . Thus, is a convex combination of .

3.3 Examples of convex sets
...

We first give some geometric examples of convex sets.

Example

Pasted image 20230917223520.png

A particular example of convex sets is the convex cone.

Definition (Conic combination)

Given , is a conic combination of if for all , .

A convex cone is a set closed under conic combinations. A convex cone hull of a set is the conic combination version of a convex hull.

Definition (Convex cone)

The convex cone hull of a set is the set of all conic combinations of points in , namely, Pasted image 20230917232913.png

Clearly, any cone is a convex set.

Another examples include , hyperplanes and halfspaces.

Example
  • Affine sets are all convex sets. So and hyperplane are convex sets.
  • A halfspace defined by (or for open halfspace) is a convex set.
    However, is not affine unless .
Proof (Convexity of halfspaces)

For all , let , where and . Since we conclude that is also in the halfspace .

Convexity is not only a property of geometric shapes.

Example (Definite matrices)

Let and denote the set of all positive semidefinite matrices and the set of all positive definite matrices, respectively, namely,Then both and are convex sets.

Proof

For all , let and ,

  1. it's easy to verify that is symmetric.
  2. , ,.
Example (Euclidean balls)

Given , the Euclidean ball is a convex set for any .

Proof

For any two points in ,

In fact, note that we do not need the norm function to be -norm. We only use the triangle inequality and the absolute homogeneity in the proof. Hence the norm balls defined by other norm functions are also convex sets.

Convexity-preserving operations
...

Example (Ellipsoid)

The Ellipsoid in is convex.

Why? An idea is to define a norm and the ellipsoid can be viewed as a norm ball. The other viewpoint is that, an ellipsoid is the image of a ball under a linear (or affine) map. To see this, note that In general, given an invertible , the set gives an ellipsoid.

Now we show that an affine map is a convexity-preserving operation.

Proposition

Suppose is a convex set, is an affine set. Then is convex.

Proof

Without loss of generality, assume for some and . Then for all , there exists such that and .
For all , since is convex, is also in . Therefore, which yields that is also convex.

Proposition (Convexity-preserving operations)

The following operations preserve the convexity:

  • (Affine map) If is convex, is an affine map, then is convex.
  • (Intersection) If and are both convex, then is also convex.
    • This property works for infinite sets intersection.
    • Unfortunately, union is not a convexity-preserving operation.
  • (Cartesian product) If and are both convex, then their cartesian product is also convex.
  • (Minkowski addition) If and are both convex, then their Minkowski addition result is also convex.
    • is also convex.

Polyhedron and polytope
...

Definition (Polyhedron and polytope)

A polyhedron (多面体) is the intersection of some halfspaces:A polytope (多胞体) is a bounded polyhedron.

Tip
  • Affine sets are polyhedra. (Because is equivalent to .)
  • Halfspaces are polyhedra.
  • Polyhedra are convex sets.

In particular, we define the simplex (单纯形) as “simplest” polytope:

  • the -simplex is just a point;
  • the -simplex is a segment;
  • the -simplex is a triangle;
  • the -simplex is a tetrahedron;
  • ……
    Specifically, a -simplex is a -dimensional polytope which is the convex hull of its  vertices. More formally, suppose the points are affinely independent. Then the simplex determined by them is their convex hull The standard simplex or probability simplex is the dimensional simplex in whose vertices are the standard unit vectors in . Namely, the standard -simplex is given by For example, the standard -simplex is the triangle whose vertices are , and .
Question

Why are simplexes polyhedra?

Suppose is a -simplex. Then if and only if there exists such that , and for all . Equivalently, we have Now let and . Clearly . Thus, can be equivalently written as Note that are linearly independent vectors (since are affinely independent). So has full rank and is invertible. Let . For any , for some . Thus , which yields that . Note that the constraints for are and . Denote by We obtain that . Overall, can be written as which gives that is a polytope.
In fact, note that and for all . This argument has a simple geometric explanation.

Table Of Contents

lec03

Lecture 3. Convex Sets

3.1 Affine sets

3.2 Convex sets

3.3 Examples of convex sets

Convexity-preserving operations

Polyhedron and polytope