dc.contributor.author | Cui, Y | |
dc.contributor.author | Liu, Z | |
dc.contributor.author | Li, Q | |
dc.contributor.author | Chan, AB | |
dc.contributor.author | Xue, CJ | |
dc.date.accessioned | 2024-07-22T09:45:52Z | |
dc.date.available | 2024-07-22T09:45:52Z | |
dc.date.issued | 2021-11-02 | |
dc.identifier.citation | Y. Cui, Z. Liu, Q. Li, A. B. Chan and C. J. Xue, "Bayesian Nested Neural Networks for Uncertainty Calibration and Adaptive Compression," 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 2392-2401, doi: 10.1109/CVPR46437.2021.00242. keywords: {Training;Computer vision;Uncertainty;Computational modeling;Neural networks;Data models;Bayes methods}, | en_US |
dc.identifier.issn | 1063-6919 | |
dc.identifier.uri | https://qmro.qmul.ac.uk/xmlui/handle/123456789/98288 | |
dc.description.abstract | Nested networks or slimmable networks are neural networks whose architectures can be adjusted instantly during testing time, e.g., based on computational constraints. Recent studies have focused on a “nested dropout” layer, which is able to order the nodes of a layer by importance during training, thus generating a nested set of sub-networks that are optimal for different configurations of resources. However, the dropout rate is fixed as a hyperparameter over different layers during the whole training process. Therefore, when nodes are removed, the performance decays in a human-specified trajectory rather than in a trajectory learned from data. Another drawback is the generated sub-networks are deterministic networks without well-calibrated uncertainty. To address these two problems, we develop a Bayesian approach to nested neural networks. We propose a variational ordering unit that draws samples for nested dropout at a low cost, from a proposed Downhill distribution, which provides useful gradients to the parameters of nested dropout. Based on this approach, we design a Bayesian nested neural network that learns the order knowledge of the node distributions. In experiments, we show that the proposed approach outperforms the nested network in terms of accuracy, calibration, and out-of-domain detection in classification tasks. It also outperforms the related approach on uncertainty-critical tasks in computer vision. | en_US |
dc.format.extent | 2392 - 2401 | |
dc.publisher | IEEE | en_US |
dc.rights | © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | |
dc.title | Bayesian Nested Neural Networks for Uncertainty Calibration and Adaptive Compression | en_US |
dc.type | Conference Proceeding | en_US |
dc.identifier.doi | 10.1109/CVPR46437.2021.00242 | |
pubs.notes | Not known | en_US |
pubs.publication-status | Published | en_US |
rioxxterms.funder | Default funder | en_US |
rioxxterms.identifier.project | Default project | en_US |
rioxxterms.funder.project | b215eee3-195d-4c4f-a85d-169a4331c138 | en_US |