How to Learn Programming for Computational Neuroscience 🧠 - Self-Study Guide

So you want to start on the amazing quest of becoming a computational neuroscientist? One of the first skills you will need to learn is programming. In computational neuroscience, programming is not just the tool that we use, but it is also a foundational element of how the brain is viewed to operate:

“The expression “computational” in computational neuroscience reflects the role of the computer as a research tool in modelling complex systems such as networks, ganglia, and brains … with the deep-seated conviction that what is being modelled by a computer is itself a kind of computer.”

The computational brain, Patricia Churchland and Terry Sejnowski

However, if you look at the resources online to learn computational neuroscience it might seem overwhelming. You need to know Psychology, Biology, Physics, Maths, even a bit of Philosophy and of course the biggest hurdle of all: You need to learn how to program. If you are wondering where to start your programming journey, you arrived at the right place.

Here, I will give you a roadmap of how to learn programming for computational neuroscience by yourself. This will not be an exhaustive list, but a good starting point for every beginner to kick off their learning journey. If you miss some resources do let me know and I will add them.

Happy learning! 🧠

👩‍💻 4 steps to programming

In this article, I will give 4 steps I would take if I were trying to learn coding from scratch. This is a method that I think works best. However, I am not saying it will be an easy ride. I think one of the fastest ways to learn to program is by embracing discomfort. It sometimes seems we can just learn programming through pre-curated courses that give us easy bite-sized chunks to learn. However, to become really good at programming we need to deep dive into the material, set up our own projects, struggle with the data etc. The real world is not clean and easy and we need to be flexible enough to handle that. One of the skills you will learn as a good programmer is the patience to keep on going even when your program has crashed for the 100th time.

So let’s go to the learning path.

📚 Step 1: Learn the basics first and fast

If you want to start computational neuroscience the first thing you will need to know is programming. However, I have seen many students spend months on learning the syntax of different languages and deep diving into different topics. In my regard, this is the wrong approach. The first thing I would do is to learn the basics of programming first and fast and then move on to step 2 as quickly as possible.

The two languages that are mainly used in computational neuroscience are R and Python. In general, I would start with Python, as it is a relatively straightforward language and has a lot of resources online to learn it with. Some basic programming skills that you will need to know are:

• Python - Python for Everybody Specialization - this specialization is a good starting point, taking you from beginner to medium programmer in a few weeks. I would take it at triple speed, as otherwise, it will take quite a long time. Another good course is 100 days of code.
• GitHub - After you have the basics of python down, I would set up a GitHub account and start learning the basic git commands, this will come in handy later when you want to show your work or share your projects.
• Command line -  I would learn how to operate the terminal a bit and get used to a few basic commands such as: ls, cd, mkdir etc. As you will be working with huge amounts of data these basic skills can help you a lot.
• Jupyter notebooks - The main tool I use for programming are notebooks. Here, you can find a good introduction on how to use notebooks. I love them for sharing code quickly and teaching other people how to code.

For this part, I would take around 2/3 months to learn these skills. Aside from the courses I recommend there are many other good resources, but the most important thing at this stage is to pick one course and finish it. If you put in the effort it is quite easy and the learning will be really rewarding as you can start making simple games etc. quite fast. After you have acquired these skills you can move on to step two 'find a topic'.

❗ Step 2: Find a topic

Now that you have the basic programming skills down it is time to find a neuroscience topic that you like and want to explore further. Computational neuroscience is a fast and complex field that seems to be quite homogenous from the outside. However, if you do a deep dive into the different areas you will quickly notice they are extremely diverse. There are different ways to start approaching computational neuroscience: from a particular area of the brain you are studying (visual cortex, motor cortex, somatosensory cortex etc.), the level of organisation you are looking at (synapses, neurons, large-scale networks etc.), the type of questions you want to answer (biological, behavioural, predictive) and the type of algorithm you are applying. Here is a non-exhaustive list of different areas in computational neuroscience from the journal Nature:

• Biophysical models
• Dynamical systems
• Learning algorithms
• Network models
• Neural decoding
• Neural encoding

Aside from these, there are many different ways to subdivide computational neuroscience. Although most computational neuroscientists have a general understanding of most of the subfields, to contribute and understand the latest research/advances in these subfields will require extensive studying. Therefore, to contribute to computational neuroscience and get a grip on the field it is good to pick one topic first and make a small research project within that topic and then move to the next topic.

If you are completely unsure which topics you like within computational neuroscience I would recommend following some of the biggest names in computational neuroscience on Twitter. They usually post their papers and papers from fellow researchers. Try reading one or two papers a day and highlighting topics that you like and find interesting. Most people will usually start reading a book or taking a course on computational neuroscience, but I believe as the field is moving so fast, that most books on computational neuroscience are already quite outdated after a few years. Following researchers is a more organic way of approaching the current literature and will give you a much better feeling of where the field is currently going.

Here is a list of 10 names that I picked randomly of researchers you can follow:

• Ole Jensen @neuosc
• Daniele Marinazzo @dan_marinazzo
• Alex Fornito@AFornito
• Fidel Alfaro Almagro @NeuroFidelity
• Linden Parks @LindenParkes
• Andrew Zalesky @AndrewZalesky
• Gael Varoquaux @GaelVaroquaux
• Olaf Sporns @spornslab
• Quentin Huys @docqhuys
• Russ Poldrack @russpoldrack

Following these researchers, you will come across a diversity of topics. I especially recommend reading the limitations section of their papers, as then you will get a grasp of possible problems in the field. To get a better feeling of the different subtopics let's look at three computational areas and how they are applied in neuroscience within three papers.

Machine learning

• Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., and Oliva, A. (2016). Comparison of deep neural networks to Spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific reports 6(1): 1-13. DOI: 10.1038/srep27755
• Skills needed - making a deep neural network, understanding and curating functional MRI data, performing a 2D surface-based searchlight analysis.

Dynamical systems

• Lindsay, G. W., Rubin, D. B., and Miller, K. D. (2019). A simple circuit model of visual cortex explains neural and behavioral aspects of attention. bioRxiv 875534. DOI: 10.1101/2019.12.13.875534
• Skills needed - understanding simple model circuits or stabilized supralinear networks (SSN),  different theories about attention effects, and convolutional neural networks.

Reinforcement learning

• Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., … and Botvinick, M. (2018). Prefrontal cortex as a meta-reinforcement learning system. Nature neuroscience 21(6): 860-868. DOI: 10.1038/s41593-018-0147-8
• Skills needed - reinforcement learning, entropy regularization, gradient descent and backpropagation.

It will take time to get an overview of all the topics out there. So I would take around a month or two to really deep dive into the literature. After you have found an area or topic you like, you can move on to the next step.

⚙️ Step 3: Find a project

This will most likely be the most important part of your learning journey. Finding a project that you can use to implement your newly learned python skills and explore the topic that you chose in neuroscience. There are several benefits to project-based learning:

• You will learn exactly what you need
• You will come in contact with the messiness of real data
• You can immediately reinforce your learning with coding
• You will have a project that you can put on your CV or Github at the end

There are a few ways to get started setting up your project. It will depend on your level of programming and familiarity with the field where it would be best to start. I will give several options ranging from beginner to more advanced:

• Neuromatch - this is an online summer school of 3 weeks where they not only take you over several areas of computational neuroscience, but also give you a project to work on. This year I worked with a group of students on an fMRI project with data from the Human Connectome Project.
• Kaggle - offers competitions and several coding challenges. Sometimes they have neuroscience-specific coding challenges, but even the challenges outside of neuroscience will be informative.
• Hackathons - throughout the year there are several hackathons in neuroscience, one example is the OHBM hackathon, but many more can be found. Hackathons are great ways to test your knowledge against others.
• Lab rotation - if you already have some idea of what you would like to do in neuroscience it could be an option to email several professors to see if they have a place for you during the year. But remember to be respectful.
• Internship - several companies offer internships and projects in neuroscience for example: the Allen institute, Google AI, Facebook Ai and TNO. These internships can be quite competitive, but if you get in it is a great way to learn a lot in a short amount of time.

Depending on your level and skill you will maybe first do a few hackathons and afterwards apply to a lab and or internship. If you already have a bachelor's in a STEM field it is usually possible to get a lab rotation straight away. If you are entirely new to STEM sciences I would first try a few summer school projects and Kaggle competitions to build up your CV and skills.

🗂 Step 4: Update your knowledge

This is the final and beginning step of your learning journey. Once you have a project you will quickly uncover the huge gaps in your knowledge. You will come to realize there is so much more to learn, both in programming and neuroscience. This is where the fun starts. Embrace being a beginner and starting fresh on a new topic. Every time you find a new term or skill that you did not know that you needed for your project you will learn that.

For example, most computational neuroscience projects that I now work on rely on these skills:

• GLM - General linear models
• Dimensionality reduction (PCA, ICA)
• Deep learning (autoencoders)
• Graph theory
• Bayesian statistics

One thing that I usually recommend my students to do is to take the machine learning specialization by Andrew Ng 👉 https://imp.i384100.net/RyjykN

Then again, for another computational neuroscience project different specialized skills may be required and that is why it is so good to first have a project. There are many resources out there to learn and upgrade your programming skills: https://github.com/dair-ai/ML-YouTube-Courses. You will probably at this point need to curate your own list specific to the skills you need to learn.

Conclusion

If you want to learn to program for computational neuroscience it will be a long and hard journey, but along the way, you will encounter and overcome many challenges. You will have to learn the basics first and fast. Afterwards, you will go on to pick a topic and project in computational neuroscience. During these projects, you will learn skills that you may not have thought were necessary before starting your learning journey. You will go step by step and then you will look back and realise you have learned so much and still so little.

Embracing being a beginner will be the biggest skill you learn. Good luck! 🧠