From Classification to Regression: Using a Fruitfly to Solve Equations

Core Methodology§

The paper introduces a regression framework inspired by the fruitfly's olfactory system, which uses a finite set of local patterns to represent complex nonlinear input-output relationships. Instead of training a single global model, the method constructs a library of representative patterns during an offline phase. For a query input $\mathbf{x}$, predictions are made by: 1. Computing similarity $s(\mathbf{x}, \mathbf{p}_i)$ between the query and each stored pattern $\mathbf{p}_i$ using a suitable kernel (e.g., Gaussian RBF). 2. Aggregating associated responses $y_i$ via weighted reconstruction: $$\hat{y}(\mathbf{x}) = \frac{\sum_{i=1}^{N} s(\mathbf{x}, \mathbf{p}_i) \cdot y_i}{\sum_{i=1}^{N} s(\mathbf{x}, \mathbf{p}_i) + \epsilon}$$ where $\epsilon$ is a small constant to avoid division by zero.

Offline-Online Workflow§

• Offline phase: Patterns are extracted from data (e.g., via clustering) or from governing equations (e.g., by sampling solution manifolds). This phase is computationally intensive but done once.
• Online phase: Given a query, only similarity computation and weighted averaging are required, making inference fast and memory-efficient. The trade-off between accuracy and cost is controlled by the number of patterns $N$ and the similarity kernel bandwidth.

Implementation Details§

Similarity Measures§

• For dynamical systems, the paper uses time-delay embeddings to capture temporal patterns.
• For physics-informed learning, they use distance metrics in the space of PDE solutions.

Pattern Library Construction§

Patterns can be selected via:

• Random sampling from the training set.
• K-means clustering of inputs or outputs.
• Sparse coding or dictionary learning.

Code Snippet (PyTorch-style)§

import torch
import torch.nn as nn

class FruitflyRegressor(nn.Module):
    def __init__(self, patterns, responses, sigma=1.0):
        super().__init__()
        self.patterns = nn.Parameter(patterns, requires_grad=False)  # (N, d)
        self.responses = nn.Parameter(responses, requires_grad=False)  # (N,)
        self.sigma = sigma

    def forward(self, x):
        # x: (B, d)
        diff = x.unsqueeze(1) - self.patterns.unsqueeze(0)  # (B, N, d)
        dist_sq = torch.sum(diff**2, dim=2)  # (B, N)
        sim = torch.exp(-dist_sq / (2 * self.sigma**2))  # (B, N)
        weights = sim / (sim.sum(dim=1, keepdim=True) + 1e-8)
        out = (weights * self.responses.unsqueeze(0)).sum(dim=1)  # (B,)
        return out

Key Results§

• The method achieves competitive performance on nonlinear dynamical system prediction (e.g., Lorenz attractor) and standard regression benchmarks.
• Explicit control over N allows practitioners to trade off accuracy for memory and speed.
• Compared to neural networks, it offers interpretability (patterns are interpretable prototypes) and simpler training (no backpropagation).

Limitations§

• Performance depends heavily on the quality of the pattern library; poor coverage leads to inaccurate predictions.
• The method is sensitive to the choice of similarity kernel and its hyperparameters.
• For high-dimensional inputs, similarity computation may become costly unless efficient approximate nearest neighbor methods are used.