On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,878 papers · 148 categories

Trend · papers per month

count % of corpus

8 results for “Mish”

Mish: A Self Regularized Non-Monotonic Activation Functioncs.LG

Mish is a new activation function that improves neural network performance.

problem Improving the performance and training dynamics of neural networks.

method Mish is a self-regularized non-monotonic activation function defined as

f(x)=x anh(softplus(x))

. It outperforms other functions on benchmarks like ImageNet-1k and MS-COCO.

result Mish outperforms Leaky ReLU and ReLU on benchmarks like MS-COCO and ImageNet-1k, respectively, with comparable network parameters.

activation function neural networks Mish YOLOv4 ResNet-50

2019-08-23abs ↗pdf ↗

U-Det: A Modified U-Net architecture with bidirectional feature network for lung nodule segmentationeess.IV

U-Det improves lung nodule segmentation in CT images.

problem Challenging shapes and surroundings of lung nodules in CT images.

method End-to-end deep learning with Bi-FPN, Mish activation, and class weights.

result U-Det achieves 82.82% Dice similarity coefficient, comparable to human experts.

lung nodule segmentation CT images deep learning Bi-FPN Mish activation

2020-03-20abs ↗pdf ↗

Deep Network Approximation: Beyond ReLU to Diverse Activation Functionscs.LG

Deep networks can approximate various activation functions with modest adjustments.

problem Expressive power of deep neural networks with diverse activation functions.

method Approximation of any activation function in set A by ReLU networks with specific scaling factors.

result Approximation of any activation function in a specific subset of A by ReLU networks with (1,1) scaling factors.

Deep Learning Neural Networks Activation Functions ReLU Approximation

2023-07-13abs ↗pdf ↗

Learning Neural Activationscs.LG

New method learns neural network activation functions from data.

problem Learning activation functions for neural networks.

method Model each neuron's activation function as a small neural network.

result Learned activation functions improve network performance.

neural networks activation functions machine learning deep learning neural activation learning

2019-12-27abs ↗pdf ↗

Universal Activation Function For Machine Learningcs.LG

A new Universal Activation Function improves performance across various machine learning tasks.

problem Achieving near optimal performance in different machine learning tasks.

method Optimization algorithms evolve the UAF's parameters to match the optimal activation function for each task.

result The UAF converges to near optimal performance in classification, quantification, and reinforcement learning tasks.

Universal Activation Function Machine Learning Optimization Classification Quantification

2020-11-07abs ↗pdf ↗

On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmeticcs.LG

Quantized neural networks can represent all fixed-point functions under certain conditions.

problem Expressive power of quantized neural networks under fixed-point arithmetic.

method Analyzing necessary and sufficient conditions for quantized networks to represent all fixed-point functions.

result Various popular activation functions satisfy the sufficient condition for representing all fixed-point functions.

quantized networks fixed-point arithmetic expressive power activation functions round-off errors

2024-08-30abs ↗pdf ↗

LSQ+: Improving low-bit quantization through learnable offsets and better initializationcs.CV

LSQ+ improves quantization of neural nets with Swish activations, achieving state-of-the-art results.

problem Quantization of neural nets with Swish activations, especially negative activations, leads to significant performance loss.

method Introduces learnable scale and offset parameters for asymmetric quantization, and uses MSE-based initialization for quantization parameters.

result Significantly outperforms LSQ for low-bit quantization of neural nets with Swish activations, achieving up to 5.6% gain with W2A2 quantization of EfficientNet-B0.

quantization neural nets Swish activations negative activations state-of-the-art

2020-04-20abs ↗pdf ↗

A Unified and Constructive Framework for the Universality of Neural Networkscs.LG

Unified framework proves neural networks' ability to mimic complex tasks.

problem Lack of a single constructive framework for neural network universality.

method Introduces neural network approximate identity (nAI) and proves it leads to universality.

result Any nAI activation function is universal.

neural networks universality activation functions approximate identity functional analysis

2021-12-30abs ↗pdf ↗