Consider the supervised learning task.

The objective function being minimized is f(w) = w · x, where w ∈ R is the parameter. Stochastic Gradient Descent with learning rate of 0.10.

Let w = 10.00 be the ith iteration(wi). The value of w at the end of iteration (i+1) is if x = 10 _______. Round off 2 decimal.

The correct answer is 9.00.

To understand why, let's break down the step-by-step application of Stochastic Gradient Descent (SGD).

Step 1: Understand the Objective Function and Parameter
We are given the objective function:
$$f(w)=w\cdot x$$
where $w$ is the parameter to be updated, and $x$ represents the input value (data point). For the given iteration, we have:
$$x=10$$

Step 2: Calculate the Gradient
Stochastic Gradient Descent updates the parameter by moving in the direction opposite to the gradient of the objective function. First, we compute the derivative (gradient) of $f(w)$ with respect to $w$:
$$f^{\prime }(w)=\frac{d}{dw}(w\cdot x)=x$$
Given that $x=10$, the gradient at the current iteration is:
$$\nabla f(w_i)=10$$

Step 3: Apply the SGD Update Rule
The formula to update the parameter in SGD is:
$$w_{i+1}=w_i-\eta \cdot \nabla f(w_i)$$
where:
• $w_i=10.00$ is the value of $w$ at the current $i$-th iteration.
• $\eta =0.10$ is the learning rate.
• $\nabla f(w_i)=10$ is the calculated gradient.

Substituting the values into the update equation:
$$w_{i+1}=10.00-(0.10\cdot 10)$$
$$w_{i+1}=10.00-1.00$$
$$w_{i+1}=9.00$$

Rounding to 2 decimal places, we get the final updated value of $w$ at the end of iteration (i+1) as 9.00.