Single Perceptron

easyFREENeural NetworksMachine Learning 2

Implement a single perceptron forward pass.

Given an input vector $x$ , weight vector $w$ , and bias $b$ , compute:

$z = w \cdot x + b$ $\text{output} = \begin{cases} 1 & \text{if } z \geq 0 \\ 0 & \text{otherwise} \end{cases}$

Process multiple input samples and return a list of outputs.

Example

Input

X = [[1, 1], [0, 1], [1, 0], [0, 0]]
w = [1, 1]
b = -1.5

Output

[1, 0, 0, 0]

Reasoning

Inputs: X holds four 2-feature samples (rows), w = [1, 1] gives each feature equal weight, and b = -1.5 shifts the decision boundary.
Weighted sum: for each sample, $z = w_1 x_1 + w_2 x_2 + b$ $z = w_{1} x_{1} + w_{2} x_{2} + b$ . With both weights equal to 1, $z$ $z$ is just the number of active features minus 1.5:
- $[1, 1]$ : $z = 1 + 1 - 1.5 = 0.5$
- $[0, 1]$ and $[1, 0]$ : $z = 1 - 1.5 = -0.5$
- $[0, 0]$ : $z = 0 - 1.5 = -1.5$
Step activation: only $z = 0.5$ satisfies $z \geq 0$ , so it maps to $1$ ; the three negative values map to $0$ , giving [1, 0, 0, 0].
Notice: this perceptron computes logical AND. Raising the bias to $-0.5$ lets a single active feature reach $z = 0.5$ , turning it into OR ([1, 1, 1, 0]). Also, $z$ exactly $0$ counts as $1$ , since the threshold is inclusive.

Constraints

X: 2D list of input vectors
w: list of weights
b: scalar bias
Return list of 0 or 1 outputs

Learn this in

Neural NetworksML Study Plan

More from Machine Learning 2

Kernel PCAhard PCA via Covariance Matrixmedium Variance Explained Ratioeasy Feature Scalingeasy GAN Discriminator Losshard

Concept primer

Background Knowledge

The single perceptron is a fundamental component of neural networks, which are a crucial part of machine learning. It is essentially a mathematical model that mimics the behavior of a single neuron in the human brain. The perceptron receives one or more inputs, performs a computation on those inputs, and then sends the output to other neurons. This process is repeated multiple times, allowing the neural network to learn complex patterns in data.

In the context of the single perceptron, the computation involves taking the dot product of the input vector $x$ and the weight vector $w$ , and then adding a bias term $b$ . The result, $z$ , is then passed through an activation function, which determines the output of the perceptron. In this case, the activation function is a simple threshold function, where the output is $1$ if $z \geq 0$ and $0$ otherwise. This type of activation function is also known as a step function or Heaviside function.

The single perceptron is a basic building block of more complex neural networks, and understanding how it works is essential for understanding deep learning concepts. The perceptron can be used for binary classification tasks, where the goal is to predict one of two classes or labels. By combining multiple perceptrons, more complex classification tasks can be performed.

Algorithm/Approach

The general approach to solving this type of problem involves the following steps:

Compute the dot product of the input vector and the weight vector
Add the bias term to the result
Apply the activation function to determine the output
Repeat this process for multiple input samples

This approach can be implemented using a variety of programming languages and libraries, including Python and NumPy.

Step-by-Step Strategy

To implement the single perceptron forward pass, follow these steps:

Define the input vector $x$ , weight vector $w$ , and bias $b$ .
Compute the dot product of $x$ and $w$ using the np.dot() function.
Add the bias $b$ to the result.
Apply the threshold function to determine the output.
Repeat steps 2-4 for multiple input samples.

Common Pitfalls

Some common pitfalls to watch out for when implementing the single perceptron include:

Forgetting to add the bias term
Using the wrong activation function
Not handling multiple input samples correctly

Time & Space Complexity

The time complexity of the single perceptron forward pass is $O(n)$ , where $n$ is the number of input samples. The space complexity is $O(n)$ as well, since we need to store the output for each input sample. Note that these complexities assume that the dot product and activation function can be computed in constant time. In practice, the actual time and space complexity may be higher due to the specifics of the implementation.

solution.py

Python 3.13

Test results