Implement a function to compute the numerical gradient of a given function at a specific point using finite differences. This task is essential in calculus and optimization when analytical gradients are not available.
The concept of gradients is crucial in understanding the rate of change of a function. In mathematics, the derivative of a function f at a point x represents the rate of change of the function with respect to x. The derivative can be approximated using the central difference formula, which is a fundamental concept in numerical analysis.
To approximate the derivative, we can follow these steps:
This technique is widely used in machine learning for training models when analytical gradients are unavailable.
numerical_gradient(lambda x: x**2, 3.0)
6.0
d/dx(x²) = 2x, at x=3: gradient = 6