📚 DAA Tutorial

Design and Analysis of Algorithms

Learn Design and Analysis of Algorithms concepts with simple explanations, algorithms, examples and practical notes.

← Back to DAA

0/1 Knapsack Problem

Given a knapsack (bag or container) with:

  • M – capacity of the knapsack
  • n – number of objects
  • w – an array consisting of the weights w1, w2, w3, …, wn
  • P – an array consisting of the profits P1, P2, P3, …, Pn
  • x – an array consisting of 0 or 1: xi = 0 means the ith object has not been selected, and xi = 1 means it has been selected.

The main objective is to place the objects into the knapsack so that the maximum profit is obtained:

Maximize∑i=1nPixisubject to∑i=1nwixi≤M

Dynamic programming for 0/1 knapsack

Let i represent the ith object in question, to be selected or not, with weight wi and profit Pi, and let j be the remaining capacity, the aim being to get the maximum profit. V[i,j] is the profit obtained by considering all i objects with capacity j.

Case 1

If there are no objects (i = 0) or the capacity of the knapsack itself is zero (j = 0), the profit is 0:

V[i,j]=0if (i = 0 or j = 0)

Case 2

V[i,j]=V[i−1,j]if wi>j

(the weight of the ith object is greater than the remaining capacity j). In this case the object is not selected, because it cannot fit into the knapsack.

Case 3

wi is less than or equal to j, so there are two alternatives:

  • Reject the ith object ⇒ profit V[i,j]=V[i−1,j]
  • Select the ith object ⇒ profit V[i,j]=V[i−1,j−wi]+Pi

Whichever yields the maximum profit is selected:

V[i,j]=max{V[i−1,j],V[i−1,j−wi]+Pi}if wi≤j

Complete recurrence

V[i,j]={ 0if i = j = 0 V[i−1,j]if wi>j max(V[i−1,j],V[i−1,j−wi]+Pi)if wi≤j

V[N,M] gives the optimal profit obtained.

Example

Apply the bottom-up dynamic programming technique to the following instance of the knapsack problem with capacity M = 5.

ItemWeightValue
12$12
21$10
33$20
42$15

Solution: N = 4, w1 = 2, w2 = 1, w3 = 3, w4 = 2, and P1 = 12, P2 = 10, P3 = 20, P4 = 15.

The solution matrix consists of N + 1 rows and M + 1 columns.

Step 1

In the bottom-up approach, start from the initial solutions i = 0 or j = 0. For i = 0 there are no objects, and for j = 0 the capacity is zero, so these entries are 0 (the profit table).

i ↓   j →012345
0000000
10
20
30
40

Step 2: i = 1, wi = 2, Pi = 12

Check whether object 1 can be selected or not.

wi vs jCalculation
2 > 1V[1,1]=V[0,1]=0
2 ≤ 2V[1,2]=max(V[0,2],V[0,0]+12)=max(0,0+12)=12
2 ≤ 3V[1,3]=max(V[0,3],V[0,1]+12)=max(0,0+12)=12
2 ≤ 4V[1,4]=max(V[0,4],V[0,2]+12)=max(0,0+12)=12
2 ≤ 5V[1,5]=max(V[0,5],V[0,3]+12)=max(0,0+12)=12

Step 3: i = 2, wi = 1, Pi = 10

wi vs jCalculation
1 ≤ 1V[2,1]=max(V[1,1],V[1,0]+10)=max(0,0+10)=10
1 ≤ 2V[2,2]=max(V[1,2],V[1,1]+10)=max(12,0+10)=12
1 ≤ 3V[2,3]=max(V[1,3],V[1,2]+10)=max(12,12+10)=22
1 ≤ 4V[2,4]=max(V[1,4],V[1,3]+10)=max(12,12+10)=22
1 ≤ 5V[2,5]=max(V[1,5],V[1,4]+10)=max(12,12+10)=22

Step 4: i = 3, wi = 3, Pi = 20

wi vs jCalculation
3 > 1V[3,1]=V[2,1]=10
3 > 2V[3,2]=V[2,2]=12
3 ≤ 3V[3,3]=max(V[2,3],V[2,0]+20)=max(22,0+20)=22
3 ≤ 4V[3,4]=max(V[2,4],V[2,1]+20)=max(22,10+20)=30
3 ≤ 5V[3,5]=max(V[2,5],V[2,2]+20)=max(22,12+20)=32

Step 5: i = 4, wi = 2, Pi = 15

wi vs jCalculation
2 > 1V[4,1]=V[3,1]=10
2 ≤ 2V[4,2]=max(V[3,2],V[3,0]+15)=max(12,0+15)=15
2 ≤ 3V[4,3]=max(V[3,3],V[3,1]+15)=max(22,10+15)=25
2 ≤ 4V[4,4]=max(V[3,4],V[3,2]+15)=max(30,12+15)=30
2 ≤ 5V[4,5]=max(V[3,5],V[3,3]+15)=max(32,22+15)=37

The completed profit table

j (capacity) → M
i ↓012345
0000000
10012121212
201012222222
301012223032
401015253037

V[N,M]=V[4,5]=37, the optimal solution (the highlighted cells are the ones visited when finding the objects selected).

Objects selected

Initially (x1,x2,x3,x4)=(0,0,0,0), which indicates that no objects are selected. The remaining capacity is j = 5.

StepCompareResultDecisionNext
1V[4, 5] = 37 and V[3, 5] = 32differentobject 4 is selected, x(4) = 1j = 5 − w4 = 5 − 2 = 3; i = i − 1 = 3, so consider V[3, 3]
2V[3, 3] = 22 and V[2, 3] = 22equalobject 3 is not selected, x(3) = 0j stays 3; i = 2
3V[2, 3] = 22 and V[1, 3] = 12differentobject 2 is selected, x(2) = 1j = 3 − w2 = 3 − 1 = 2; i = 1, so consider V[1, 2]
4V[1, 2] = 12 and V[0, 2] = 0differentobject 1 is selected, x(1) = 1j = 2 − w1 = 2 − 2 = 0; i = 0
⇒(x1,x2,x3,x4)=(1,1,0,1)

Check: profit = 12 + 10 + 15 = 37, and weight = 2 + 1 + 2 = 5 ≤ M.

Algorithm KNAPSACK(n, m, w, P, V)

ALGORITHM KNAPSACK(n, m, w, P, V)
/* To find the optimal solution for the knapsack problem using
   dynamic programming.
   n - number of objects
   m - capacity of the knapsack
   w - weights of all the objects
   P - profits of all the objects
   Output: V - optimal solution */
for i ← 0 to n do
    for j ← 0 to m do
        if (i = 0 or j = 0)
            V[i,j] = 0
        else if (w[i] > j)
            V[i,j] = V[i-1,j]
        else
            V[i,j] = max(V[i-1,j], V[i-1,j-w[i]] + P[i])
        end if
    end for
end for
return