Suggested further readings#

Pytorch - papers:#

Automatic differentiation library; some tutorials openreview.net/pdf?id=BJJsrmfCZ.

Intro:#

Large list of papers comparing DNNs and the brain:

Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., and Oliva, A. (2016). Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific reports 6(1): 1-13. doi: 10.1038/srep27755 Open Access publication.

Hasson, U., Nastase, S. A., & Goldstein, A. (2020). Direct fit to nature: an evolutionary perspective on biological and artificial neural networks. Neuron 105(3): 416-434. doi: 10.1016/j.neuron.2019.12.002 Open Access publication.

Heuer, K., Gulban, O. F., Bazin, P. L., Osoianu, A., Valabregue, R., Santin, M., … and Toro, R. (2019). Evolution of neocortical folding: A phylogenetic comparative analysis of MRI from 34 primate species. Cortex 118: 275-291. doi: 10.1016/j.cortex.2019.04.011 Open Access publication.

Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2015). Object detectors emerge in deep scene cnns. In ICLR, San Diego, CA, USA. arXiv:1412.6856.

Zhou, B., Bau, D., Oliva, A., and Torralba, A. (2018). Interpreting deep visual representations via network dissection. IEEE transactions on pattern analysis and machine intelligence 41(9): 2131-2145. doi: 10.1109/TPAMI.2018.2858759 Open Access publication.

Tutorials:#

Dataset:#

Stringer, C., Michaelos, M., Tsyboulski, D., Lindo, S. E., and Pachitariu, M. (2021). High-precision coding in visual cortex. Cell 184(10): 2767–2778.e15. doi: 10.1016/j.cell.2021.03.042 Closed Access publication (preprint: www.biorxiv.org/content/biorxiv/early/2019/11/04/679324.full.pdf Open Access publication).

Deep learning used for encoding models:#

Batty, E., Merel, J., Brackbill, N., Heitman, A., Sher, A., Litke, A., … and Paninski, L. (2017). Multilayer recurrent network models of primate retinal ganglion cell responses. ICLR 2017, Toulon, France. url: openreview.net/forum?id=HkEI22jeg

Cadena, S. A., Denfield, G. H., Walker, E. Y., Gatys, L. A., Tolias, A. S., Bethge, M., and Ecker, A. S. (2019). Deep convolutional models improve predictions of macaque V1 responses to natural images. PLoS computational biology 15(4): e1006897. doi: 10.1371/journal.pcbi.1006897 Open Access publication.

McIntosh, L., Maheswaranathan, N., Nayebi, A., Ganguli, S., and Baccus, S. (2016). Deep learning models of the retinal response to natural scenes. Advances in neural information processing systems, 29. url: papers.nips.cc/paper/2016/hash/a1d33d0dfec820b41b54430b50e96b5c-Abstract.html

Walker, E. Y., Sinz, F. H., Cobos, E., Muhammad, T., Froudarakis, E., Fahey, P. G., … and Tolias, A. S. (2019). Inception loops discover what excites neurons most using deep predictive models. Nature neuroscience 22(12): 2060-2065. doi: 10.1038/s41593-019-0517-x Closed Access publication.

Comparing deep networks and the brain:#

Guclu, U., and van Gerven, M. A. (2015). Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience 35(27): 10005-10014. doi: 10.1523/JNEUROSCI.5023-14.2015 Open Access publication.

Khaligh-Razavi, S. M., and Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS computational biology 10(11):e1003915. doi: 10.1371/journal.pcbi.1003915 Open Access publication.

Kriegeskorte, N., Mur, M., and Bandettini, P. A. (2008). Representational similarity analysis-connecting the branches of systems neuroscience. Frontiers in systems neuroscience 2:4. doi: 10.3389/neuro.06.004.2008 Open Access publication.

Mohsenzadeh, Y., Mullin, C., Lahner, B., and Oliva, A. (2020). Emergence of Visual center-periphery Spatial organization in Deep convolutional neural networks. Scientific Reports 10(1): 1-8. doi: 10.1038/s41598-020-61409-0 Open Access publication.

Yamins, D. L., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., and DiCarlo, J. J. (2014). Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences 111(23): 8619-8624. doi: 10.1073/pnas.1403112111 Closed Access publication (postprint: europepmc.org/articles/pmc4060707?pdf=render Open Access publication).

Deep learning:#

Goh, G. (2017). Why momentum really works. Distill 2(4): e6. doi: 10.23915/distill.00006 Open Access publication.

He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). url: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7780459

Ioffe, S., and Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp. 448-456). PMLR. url: proceedings.mlr.press/v37/ioffe15.html

Li, H., Xu, Z., Taylor, G., Studer, C., and Goldstein, T. (2018). Visualizing the loss landscape of neural nets. Advances in neural information processing systems, 31. url: papers.nips.cc/paper/2018/hash/a41b3bb3e6b050b6c9067c67f663b915-Abstract.html

Nielsen, M. (2016). A visual proof that neural nets can compute any function. url: neuralnetworksanddeeplearning.com/chap4.html.

Olah, C. (2014). Conv nets: A modular perspective. url: colah.github.io/posts/2014-07-Conv-Nets-Modular.

Outro 1#

Jozwik, K. M., Kriegeskorte, N., Storrs, K. R., and Mur, M. (2017). Deep convolutional neural networks outperform feature-based but not categorical models in explaining object similarity judgments. Frontiers in psychology 8: 1726. doi: 10.3389/fpsyg.2017.01726 Open Access publication.

Kriegeskorte, N., and Douglas, P. K. (2018). Cognitive computational neuroscience. Nature neuroscience 21(9): 1148-1160. doi: 10.1038/s41593-018-0210-5 Closed Access publication (postprint: europepmc.org/articles/pmc6706072?pdf=render Open Access publication).

Kietzmann, T. C., Spoerer, C. J., Sörensen, L. K., Cichy, R. M., Hauk, O., and Kriegeskorte, N. (2019). Recurrence is required to capture the representational dynamics of the human visual system. Proceedings of the National Academy of Sciences 116(43): 21854-21863. doi: 10.1073/pnas.1905544116 Open Access publication.

Kubilius, J., Schrimpf, M., Kar, K., Rajalingham, R., Hong, H., Majaj, N., … and DiCarlo, J. J. (2019). Brain-like object recognition with high-performing shallow recurrent ANNs. Advances in neural information processing systems, 32. url: NIPS2019

Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J., and Hinton, G. (2020). Backpropagation and the brain. Nature reviews. Neuroscience 21(6): 335–346. doi: 10.1038/s41583-020-0277-3 Closed Access publication (preprint: ora.ox.ac.uk/objects/uuid:862189c1-0088-4f78-b17a-2748c2019209/download_file?safe_filename=Lillicrap_v6_2020.pdf&file_format=pdf&type_of_work=Journal+article Open Access publication).

Nili, H., Wingfield, C., Walther, A., Su, L., Marslen-Wilson, W., and Kriegeskorte, N. (2014). A toolbox for representational similarity analysis. PLoS computational biology 10(4): e1003553. doi: 10.1371/journal.pcbi.1003553 Open Access publication.

Schrimpf, M., Kubilius, J., Hong, H., Majaj, N. J., Rajalingham, R., Issa, E. B., …, and DiCarlo, J. J. (2020). Brain-score: Which artificial neural network for object recognition is most brain-like?. bioRxiv 407007. doi: 10.1101/407007 Open Access publication.

Spoerer, C. J., Kietzmann, T. C., Mehrer, J., Charest, I., and Kriegeskorte, N. (2020). Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision. PLoS computational biology 16(10): e1008215. doi: 10.1371/journal.pcbi.1008215 Open Access publication.

Storrs, K. R., Kietzmann, T. C., Walther, A., Mehrer, J., and Kriegeskorte, N. (2021). Diverse deep neural networks all predict human inferior temporal cortex well, after training and fitting. Journal of Cognitive Neuroscience 33(10): 2044-2064. doi: 10.1162/jocn_a_01755 Closed Access publication (postprint: repository.ubn.ru.nl/bitstream/handle/2066/237374/1/237374.pdf Open Access publication).

Tang, H., Schrimpf, M., Lotter, W., Moerman, C., Paredes, A., Caro, J. O., … and Kreiman, G. (2018). Recurrent computations for visual pattern completion. Proceedings of the National Academy of Sciences 115(35): 8835-8840. doi: 10.1073/pnas.1719397115 Open Access publication.

Outro 2#

Chambers, C., Seethapathi, N., Saluja, R., Loeb, H., Pierce, S. R., Bogen, D. K., … and Kording, K. P. (2020). Computer vision to automatically assess infant neuromotor risk. IEEE Transactions on Neural Systems and Rehabilitation Engineering 28(11): 2431-2442. doi: 10.1109/TNSRE.2020.3029121 Closed Access publication (postprint: www.ncbi.nlm.nih.gov/pmc/articles/PMC8011647 Open Access publication).