Attention mechanism combines bottom-up and top-down signals in neural networks.
problem Combining robust perception with bottom-up and top-down signals.
method Attention mechanism over modulated recurrent neural networks.
result Bidirectional information flow leads to improved performance in various tasks.
BUSTLE synthesizes programs by learning from intermediate values.
problem Challenges in synthesizing complex programs due to large search space.
method Bottom-up search guided by a neural network trained on input-output examples.
result Bottom-up search with execution of intermediate programs provides valuable semantic information.
Bottom-up algorithms outperform top-down in hierarchical community detection at intermediate levels.
problem Finding the optimal hierarchical community structure in networks.
method A bottom-up algorithm for hierarchical clustering of networks.
result Bottom-up algorithms achieve the information-theoretic threshold for exact recovery at intermediate levels of the hierarchy.
CrossBeam learns to search more efficiently in program synthesis.
problem Efficiently searching through vast program spaces.
method Trains a neural model to guide program synthesis, combining previously explored programs.
result CrossBeam explores much smaller portions of the program space compared to state-of-the-art methods.
Novel non-parametric tree model learns tree distributions.
problem Learning distributions for tree-structured data.
method Bottom-up hidden tree Markov model with infinite states.
result Novel non-parametric generalization of hidden tree Markov model.
Efficient unsupervised training and inference in deep generative models remains a challenging problem. One basic approach, called Helmholtz machine, involves training a top-down directed generative model together with a bottom-up auxiliary model used for approximate inference. Recent results indicate that better genera…
Scalable subspace clustering for high-dimensional data.
problem Finding clusters in non-disjoint subspaces and scaling to large data.
method Bottom-up strategy using FP-trees for frequent pattern mining.
result The proposed algorithm produces clusters with high accuracy and scales well to large data.
We propose a novel adaptive approximation approach for test-time resource-constrained prediction. Given an input instance at test-time, a gating function identifies a prediction model for the input among a collection of models. Our objective is to minimize overall average cost without sacrificing accuracy. We learn gat…
New method restores source features for SFDA without source data.
problem Domain adaptation without access to source data.
method Feature Restoration (FR) and Bottom-Up Feature Restoration (BUFR).
result BUFR outperforms existing SFDA methods in accuracy, calibration, and data efficiency.
Bayesian tensor factorization approximates a complex tree model.
problem Intractable size of state-transition matrix in Hidden Tree Markov Models.
method Tucker factorization of tensors for probabilistic interpretation.
result New model outperforms existing approximations on tree-structured data tasks.
A generative model is developed for deep (multi-layered) convolutional dictionary learning. A novel probabilistic pooling operation is integrated into the deep model, yielding efficient bottom-up (pretraining) and top-down (refinement) probabilistic learning. Experimental results demonstrate powerful capabilities of th…
System designs for analyzing and pricing non-performing consumer credit portfolios.
problem Technical challenges in analyzing and pricing portfolios of non-performing consumer credit loans.
method Bottom-up architecture, simultaneous quantile regression, R-copula, Gaussian one-factor copula model.
result Successfully developed a methodology for analyzing credit portfolio risks of consumer loans.
We present a dynamic model selection approach for resource-constrained prediction. Given an input instance at test-time, a gating function identifies a prediction model for the input among a collection of models. Our objective is to minimize overall average cost without sacrificing accuracy. We learn gating and predict…
Multilayer bootstrap network builds a gradually narrowed multilayer nonlinear network from bottom up for unsupervised nonlinear dimensionality reduction. Each layer of the network is a nonparametric density estimator. It consists of a group of k-centroids clusterings. Each clustering randomly selects data points with r…
Efficiently learns linear non-Gaussian DAGs with noisy nodes.
problem Learning DAGs with non-Gaussian noise and diverging number of nodes.
method Proposes a novel method using topological layers for bottom-up reconstruction and consistent parent-child relations.
result Topological layers can be exactly reconstructed and parent-child relations established without faithfulness assumption.
Study improves forecasting of aggregated curves in electricity markets.
problem Improving accuracy in predicting aggregated curves like demand and supply in electricity markets.
method Exploits hierarchical structure of aggregated curves, uses reconciliation methods (bottom-up, top-down, linear optimal, aggregated-down).
result Hierarchical reconciliation methods can significantly improve forecast accuracy of aggregated curves.
NFM improves deep learning by selectively processing hidden states.
problem Processing entire hidden states in each layer limits modularity and reusability.
method Introduces Neural Function Modules (NFM) with attention, sparsity, and feedback.
result Improves results in classification, generalization, generative modeling, and reinforcement learning.
We study analytically and numerically Minsky instability as a combination of top-down, bottom-up and peer-to-peer positive feedback loops. The peer-to-peer interactions are represented by the links of a network formed by the connections between firms, contagion leading to avalanches and percolation phase transitions pr…
We show that a generative random field model, which we call generative ConvNet, can be derived from the commonly used discriminative ConvNet, by assuming a ConvNet for multi-category classification and assuming one of the categories is a base category generated by a reference distribution. If we further assume that the…
Hybrid method forecasts distribution feeder loads using LSTM and GRU networks.
problem Forecasting annual load of distribution feeders.
method Hybrid modeling combining LSTM and GRU networks for multi-year data.
result Proposed method outperforms traditional models in real-world application.
A new multi-phase approach improves supply chain forecasting accuracy.
problem Improving forecast accuracy for hierarchical supply chain demands.
method Independent child-level forecasting followed by parent-level estimation.
result 82-90% improvement in forecast accuracy compared to traditional methods.
Panoptic-DeepLab achieves state-of-the-art results in panoptic segmentation.
problem Panoptic segmentation challenges in computer vision.
method Bottom-up, single-shot approach with dual-ASPP and dual-decoder structures.
result Panoptic-DeepLab sets new state-of-the-art results on Cityscapes benchmarks.
Extends PCVM for multi-class classification with improved accuracy.
problem Lack of probabilistic outputs and contradictory predictions in multi-class classification.
method Proposes mPCVM with two learning algorithms: top-down and bottom-up.
result Superior performance, especially with many classes, validated on synthetic and benchmark data.
In this paper, we propose a novel generative model named Stacked Generative Adversarial Networks (SGAN), which is trained to invert the hierarchical representations of a bottom-up discriminative network. Our model consists of a top-down stack of GANs, each learned to generate lower-level representations conditioned on …
A generative Bayesian model is developed for deep (multi-layer) convolutional dictionary learning. A novel probabilistic pooling operation is integrated into the deep model, yielding efficient bottom-up and top-down probabilistic learning. After learning the deep convolutional dictionary, testing is implemented via dec…
Study explores robust Orlicz spaces in finance, showing separability implications.
problem Understanding robustness in financial and economic contexts.
method Distinguished two constructions of robust Orlicz spaces: top-down and bottom-up.
result Separability of robust Orlicz spaces has strong implications for dominatedness and order completeness.
We introduce a new Bayesian model for hierarchical clustering based on a prior over trees called Kingman's coalescent. We develop novel greedy and sequential Monte Carlo inferences which operate in a bottom-up agglomerative fashion. We show experimentally the superiority of our algorithms over others, and demonstrate o…
Paper uses 2-step Gradient Boosting to predict VAT tax gap.
problem Estimating tax evasion and revenue loss from tax avoidance.
method 2-steps Gradient Boosting model to correct selection bias.
result Significantly improved prediction of VAT tax gap.
LIBRE learns interpretable Boolean rules from data.
problem Creating interpretable classifiers in imbalanced settings.
method Ensemble of weak learners on random feature subsets, combined with a simple union.
result Efficiently balances prediction accuracy and interpretability.
Deep neural networks blend mechanistic and phenomenological NLP approaches.
problem Combining theory-driven and data-driven NLP methods.
method Using deep neural networks to integrate mechanistic and phenomenological models.
result Deep learning can effectively model language and perception in spatial cognition.
The paper introduces a method to make neural networks more robust to adversarial attacks.
problem Vulnerability of deep neural networks to small, adversarially designed perturbations.
method A bottom-up strategy using a nonlinear front end that polarizes and quantizes data.
result The approach can completely eliminate adversarial perturbations on MNIST and Fashion MNIST datasets.
Model learns collective and individual dynamics in time series data.
problem Lack of models capturing system-level collective behavior in individual time series.
method Hierarchical switching-state model with latent system-level and entity-level Markov chains.
result Model improves interpretability and forecasting accuracy compared to larger models.
New protocol benchmarks deep learning methods' data efficiency.
problem Measuring data efficiency of deep learning methods.
method Proposed experimental protocol to benchmark CNNs and HiGSFA.
result HiGSFA outperforms CNNs on smaller datasets.
We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with p…
TF-GNN simplifies graph neural networks in TensorFlow.
problem Handling rich heterogeneous graph data in machine learning.
method A scalable library with a Keras message passing API.
result Enables low-code solutions for broader developers.
BOinG optimizes HPO problems by focusing on promising local regions.
problem Expensive black-box optimization problems, especially hyperparameter optimization.
method Two-stage approach: global surrogate model followed by local model.
result BOinG exploits the structure of typical HPO problems and performs well on mid-sized problems.
Novel ML approach optimizes large portfolios without covariance matrix issues.
problem Static and dynamic portfolio optimization for many assets.
method Machine learning for constrained optimization, avoiding covariance matrix computation.
result Significant excess returns in U.S. and China equity markets.
Paper introduces TEP to better model treatment effect heterogeneity.
problem Personalised decision making requires evidence of treatment suitability.
method Designs TEP to represent treatment effect heterogeneity, uses local causal structure to show important variables, derives formula for unbiased CATE estimation.
result Proposed method models treatment effect heterogeneity better than existing methods.
We establish conditions for compositional generalization in machine learning.
problem Achieving compositional generalization in machine learning models.
method We reformulate compositionality as a property of the data-generating process and derive mild conditions on the training distribution and model architecture.
result Our theoretical framework enables compositional generalization under mild conditions.
We consider a set of probabilistic functions of some input variables as a representation of the inputs. We present bounds on how informative a representation is about input data. We extend these bounds to hierarchical representations so that we can quantify the contribution of each layer towards capturing the informati…
We learn hierarchical slate representations for collaborative filtering.
problem Building models for recommendation systems with hierarchical slates.
method Learning low-dimensional embeddings of hierarchical slates using recursive composition rules.
result Improved recommendation system performance on a real-world dataset.
New neural model learns from sequences without time backpropagation.
problem Learning useful temporal generative models from sequential data.
method Developed Temporal Neural Coding Network and Discrepancy Reduction algorithm based on predictive coding.
result Algorithm shows promise on bouncing balls generative modeling problem.
ASLA learns atomic structures using neural networks and reinforcement learning.
problem Designing materials and drugs with desired properties.
method Atomistic structure learning algorithm (ASLA) using a convolutional neural network and reinforcement learning.
result ASLA can predict optimal structural arrangements of atoms for various target properties.
Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that…
A new machine-learned CG model predicts protein structures efficiently.
problem Developing a universal, computationally efficient protein simulation model.
method Combining deep learning with all-atom protein simulations to create a transferable CG force field.
result The model predicts protein structures, intermediates, and fluctuations efficiently.
RICH models scenes as hierarchical tree to learn and generate complex compositions.
problem Learning compositional structures between parts and objects in natural scenes.
method RICH uses a latent scene graph to organize entities into a tree structure and employs a top-down inference approach.
result RICH learns and generates complex scene hierarchies from unlabeled data.
We describe a bottom-up framework, based on the identification of appropriate order parameters and determination of phase diagrams, for understanding progressively refined agent-based models and simulations of financial markets. We illustrate this framework by starting with a deterministic toy model, whereby N indepe…
New model simulates stock market microstructure with learning agents.
problem Lack of realistic agent learning in past financial models.
method Designed a next-generation MAS stock market simulator with model-free reinforcement learning.
result Model can faithfully reproduce market microstructure metrics.