Single Perceptron
Implement a single perceptron forward pass.
Given an input vector x, weight vector w, and bias b, compute:
z=w⋅x+b output={10​if z≥0otherwise​
Process multiple input samples and return a list of outputs.
Example:
X = [[1, 1], [0, 1], [1, 0], [0, 0]] w = [1, 1] b = -1.5
[1, 0, 0, 0]
- We calculate z for each input sample by taking the dot product of the input vector x and weight vector w, then adding the bias b. For the first sample, this gives z=(1⋅1)+(1⋅1)−1.5=0.5.
- We apply the activation function to each z value: for the first sample, since z=0.5≥0, the output is 1. For the other samples:
- x=[0,1]: z=(0⋅1)+(1⋅1)−1.5=−0.5, output is 0
- x=[1,0]: z=(1⋅1)+(0⋅1)−1.5=−0.5, output is 0
- x=[0,0]: z=(0⋅1)+(0⋅1)−1.5=−1.5, output is 0
- The final output is a list of these results: [1,0,0,0]
Constraints:
- X: 2D list of input vectors
- w: list of weights
- b: scalar bias
- Return list of 0 or 1 outputs
Background Knowledge
The single perceptron is a fundamental component of neural networks, which are a crucial part of machine learning. It is essentially a mathematical model that mimics the behavior of a single neuron in the human brain. The perceptron receives one or more inputs, performs a computation on those inputs, and then sends the output to other neurons. This process is repeated multiple times, allowing the neural network to learn complex patterns in data.
In the context of the single perceptron, the computation involves taking the dot product of the input vector x and the weight vector w, and then adding a bias term b. The result, z, is then passed through an activation function, which determines the output of the perceptron. In this case, the activation function is a simple threshold function, where the output is 1 if z≥0 and 0 otherwise. This type of activation function is also known as a step function or Heaviside function.
The single perceptron is a basic building block of more complex neural networks, and understanding how it works is essential for understanding deep learning concepts. The perceptron can be used for binary classification tasks, where the goal is to predict one of two classes or labels. By combining multiple perceptrons, more complex classification tasks can be performed.
Algorithm/Approach
The general approach to solving this type of problem involves the following steps:
- Compute the dot product of the input vector and the weight vector
- Add the bias term to the result
- Apply the activation function to determine the output
- Repeat this process for multiple input samples
This approach can be implemented using a variety of programming languages and libraries, including Python and NumPy.
Step-by-Step Strategy
To implement the single perceptron forward pass, follow these steps:
- Define the input vector x, weight vector w, and bias b.
- Compute the dot product of x and w using the np.dot() function.
- Add the bias b to the result.
- Apply the threshold function to determine the output.
- Repeat steps 2-4 for multiple input samples.
Common Pitfalls
Some common pitfalls to watch out for when implementing the single perceptron include:
- Forgetting to add the bias term
- Using the wrong activation function
- Not handling multiple input samples correctly
Time & Space Complexity
The time complexity of the single perceptron forward pass is O(n), where n is the number of input samples. The space complexity is O(n) as well, since we need to store the output for each input sample. Note that these complexities assume that the dot product and activation function can be computed in constant time. In practice, the actual time and space complexity may be higher due to the specifics of the implementation.