Skip to content

Create categorical_cross_entropy.py #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 9, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions machine_learning/loss_functions/categorical_cross_entropy.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
"""
Categorical Cross-Entropy Loss

This function calculates the Categorical Cross-Entropy Loss between true class
labels and predicted class probabilities.

Formula:
Categorical Cross-Entropy Loss = -Σ(y_true * log(y_pred))

Resources:
- [Wikipedia - Cross entropy](https://en.wikipedia.org/wiki/Cross_entropy)
"""

import numpy as np

def categorical_crossentropy(
y_true: np.ndarray, y_pred: np.ndarray, epsilon: float = 1e-15
) -> float:
"""
Calculate Categorical Cross-Entropy Loss between true class labels and
predicted class probabilities.

Parameters:
- y_true: True class labels (one-hot encoded) as a NumPy array.
- y_pred: Predicted class probabilities as a NumPy array.
- epsilon: Small constant to avoid numerical instability.

Returns:
- ce_loss: Categorical Cross-Entropy Loss as a floating-point number.

Example:
>>> true_labels = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]])
>>> pred_probs = np.array([[0.9, 0.1, 0.0], [0.2, 0.7, 0.1], [0.0, 0.1, 0.9]])
>>> categorical_crossentropy(true_labels, pred_probs)
0.18913199175146167

>>> y_true = np.array([[1, 0], [0, 1]])
>>> y_pred = np.array([[0.9, 0.1, 0.0], [0.2, 0.7, 0.1]])
>>> categorical_crossentropy(y_true, y_pred)
Traceback (most recent call last):
...
ValueError: Input arrays must have the same length.
"""
if y_true.shape != y_pred.shape:
raise ValueError("Input arrays must have the same length.")

# Clip predicted probabilities to avoid log(0)
y_pred = np.clip(y_pred, epsilon, 1 - epsilon)

# Calculate categorical cross-entropy loss
return -np.sum(y_true * np.log(y_pred)) / len(y_true)

if __name__ == "__main__":
import doctest
doctest.testmod()