{ "cells": [ { "cell_type": "markdown", "id": "dfd443f6", "metadata": { "colab_type": "text", "execution": {}, "id": "view-in-github" }, "source": [ "  " ] }, { "cell_type": "markdown", "id": "08842220", "metadata": { "execution": {} }, "source": [ "# Tutorial 3: Numerical Methods\n", "\n", "**Week 0, Day 3: Calculus**\n", "\n", "**Content creators:** John S Butler, Arvind Kumar with help from Harvey McCone\n", "\n", "**Content reviewers:** Swapnil Kumar, Matthew McCann\n", "\n", "**Production editors:** Matthew McCann, Ella Batty" ] }, { "cell_type": "markdown", "id": "7d80998b", "metadata": { "execution": {} }, "source": [ "---\n", "# Tutorial Objectives\n", "\n", "*Estimated timing of tutorial: 70 minutes*\n", "\n", "While a great deal of neuroscience can be described by mathematics, a great deal of the mathematics used cannot be solved exactly. This might seem very odd that you can writing something in mathematics that cannot be immediately solved but that is the beauty and mystery of mathematics. To side step this issue we use Numerical Methods to estimate the solution.\n", "\n", "In this tutorial, we will look at the Euler method to estimate the solution of a few different differential equations: the population equation, the Leaky Integrate and Fire model and a simplified version of the Wilson-Cowan model which is a system of differential equations.\n", "\n", "**Steps:**\n", "- Code the Euler estimate of the Population Equation;\n", "- Investigate the impact of time step on the error of the numerical solution;\n", "- Code the Euler estimate of the Leaky Integrate and Fire model for a constant input;\n", "- Visualize and listen to the response of the integrate for different inputs;\n", "- Apply the Euler method to estimate the solution of a system of differential equations." ] }, { "cell_type": "markdown", "id": "bwMygnJgMUp7", "metadata": { "execution": {} }, "source": [ "---\n", "# Setup" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Install and import feedback gadget\n" ] }, { "cell_type": "code", "execution_count": null, "id": "BrceoWcPfYlB", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Install and import feedback gadget\n", "\n", "!pip3 install vibecheck datatops --quiet\n", "\n", "from vibecheck import DatatopsContentReviewContainer\n", "def content_review(notebook_section: str):\n", " return DatatopsContentReviewContainer(\n", " \"\", # No text prompt\n", " notebook_section,\n", " {\n", " \"url\": \"https://pmyvdlilci.execute-api.us-east-1.amazonaws.com/klab\",\n", " \"name\": \"neuromatch-precourse\",\n", " \"user_key\": \"8zxfvwxw\",\n", " },\n", " ).render()\n", "\n", "\n", "feedback_prefix = \"W0D4_T3\"" ] }, { "cell_type": "code", "execution_count": null, "id": "a5985801", "metadata": { "execution": {} }, "outputs": [], "source": [ "# Imports\n", "import numpy as np\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Figure Settings\n" ] }, { "cell_type": "code", "execution_count": null, "id": "d40abcd8", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Figure Settings\n", "import logging\n", "logging.getLogger('matplotlib.font_manager').disabled = True\n", "import IPython.display as ipd\n", "from matplotlib import gridspec\n", "import ipywidgets as widgets # interactive display\n", "from ipywidgets import Label\n", "%config InlineBackend.figure_format = 'retina'\n", "# use NMA plot style\n", "plt.style.use(\"https://raw.githubusercontent.com/NeuromatchAcademy/content-creation/main/nma.mplstyle\")\n", "my_layout = widgets.Layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plotting Functions\n" ] }, { "cell_type": "code", "execution_count": null, "id": "hwhtoyMMi7qj", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Plotting Functions\n", "\n", "time = np.arange(0, 1, 0.01)\n", "\n", "def plot_slope(dt):\n", " \"\"\"\n", " Args:\n", " dt : time-step\n", " Returns:\n", " A figure of an exponential, the slope of the exponential and the derivative exponential\n", " \"\"\"\n", "\n", " t = np.arange(0, 5+0.1/2, 0.1)\n", "\n", " with plt.xkcd():\n", "\n", " fig = plt.figure(figsize=(6, 4))\n", " # Exponential\n", " p = np.exp(0.3*t)\n", " plt.plot(t, p, label='y')\n", " # slope\n", " plt.plot([1, 1+dt], [np.exp(0.3*1), np.exp(0.3*(1+dt))],':og',label=r'$\\frac{y(1+\\Delta t)-y(1)}{\\Delta t}$')\n", " # derivative\n", " plt.plot([1, 1+dt], [np.exp(0.3*1), np.exp(0.3*(1))+dt*0.3*np.exp(0.3*(1))],'-k',label=r'$\\frac{dy}{dt}$')\n", " plt.legend()\n", " plt.plot(1+dt, np.exp(0.3*(1+dt)), 'og')\n", " plt.ylabel('y')\n", " plt.xlabel('t')\n", " plt.show()\n", "\n", "\n", "\n", "def plot_StepEuler(dt):\n", " \"\"\"\n", " Args:\n", " dt : time-step\n", " Returns:\n", " A figure of one step of the Euler method for an exponential growth function\n", " \"\"\"\n", "\n", " t=np.arange(0, 1 + dt + 0.1 / 2, 0.1)\n", "\n", " with plt.xkcd():\n", " fig = plt.figure(figsize=(6,4))\n", " p=np.exp(0.3*t)\n", " plt.plot(t,p)\n", " plt.plot([1,],[np.exp(0.3*1)],'og',label='Known')\n", " plt.plot([1,1+dt],[np.exp(0.3*1),np.exp(0.3*(1))+dt*0.3*np.exp(0.3*1)],':g',label=r'Euler')\n", " plt.plot(1+dt,np.exp(0.3*(1))+dt*0.3*np.exp(0.3*1),'or',label=r'Estimate $p_1$')\n", " plt.plot(1+dt,p[-1],'bo',label=r'Exact $p(t_1)$')\n", " plt.vlines(1+dt,np.exp(0.3*(1))+dt*0.3*np.exp(0.3*1),p[-1],colors='r', linestyles='dashed',label=r'Error $e_1$')\n", " plt.text(1+dt+0.1,(np.exp(0.3*(1))+dt*0.3*np.exp(0.3*1)+p[-1])/2,r'$e_1$')\n", " plt.legend()\n", " plt.ylabel('Population (millions)')\n", " plt.xlabel('time(years)')\n", " plt.show()\n", "\n", "def visualize_population_approx(t, p):\n", " fig = plt.figure(figsize=(6, 4))\n", " plt.plot(t, np.exp(0.3*t), 'k', label='Exact Solution')\n", "\n", " plt.plot(t, p,':o', label='Euler Estimate')\n", " plt.vlines(t, p, np.exp(0.3*t),\n", " colors='r', linestyles='dashed', label=r'Error $e_k$')\n", "\n", " plt.ylabel('Population (millions)')\n", " plt.legend()\n", " plt.xlabel('Time (years)')\n", " plt.show()\n", "\n", "## LIF PLOT\n", "def plot_IF(t, V, I, Spike_time):\n", " \"\"\"\n", " Args:\n", " t : time\n", " V : membrane Voltage\n", " I : Input\n", " Spike_time : Spike_times\n", " Returns:\n", " figure with three panels\n", " top panel: Input as a function of time\n", " middle panel: membrane potential as a function of time\n", " bottom panel: Raster plot\n", " \"\"\"\n", "\n", " with plt.xkcd():\n", " fig = plt.figure(figsize=(12,4))\n", " gs = gridspec.GridSpec(3, 1, height_ratios=[1, 4, 1])\n", " # PLOT OF INPUT\n", " plt.subplot(gs)\n", " plt.ylabel(r'$I_e(nA)$')\n", " plt.yticks(rotation=45)\n", " plt.plot(t,I,'g')\n", " #plt.ylim((2,4))\n", " plt.xlim((-50,1000))\n", " # PLOT OF ACTIVITY\n", " plt.subplot(gs)\n", " plt.plot(t,V,':')\n", " plt.xlim((-50,1000))\n", " plt.ylabel(r'$V(t)$(mV)')\n", " # PLOT OF SPIKES\n", " plt.subplot(gs)\n", " plt.ylabel(r'Spike')\n", " plt.yticks([])\n", " plt.scatter(Spike_time,1*np.ones(len(Spike_time)), color=\"grey\", marker=\".\")\n", " plt.xlim((-50,1000))\n", " plt.xlabel('time(ms)')\n", " plt.show()\n", "\n", "def plot_rErI(t, r_E, r_I):\n", " \"\"\"\n", " Args:\n", " t : time\n", " r_E : excitation rate\n", " r_I : inhibition rate\n", "\n", " Returns:\n", " figure of r_I and r_E as a function of time\n", "\n", " \"\"\"\n", " with plt.xkcd():\n", " fig = plt.figure(figsize=(6,4))\n", " plt.plot(t,r_E,':',color='b',label=r'$r_E$')\n", " plt.plot(t,r_I,':',color='r',label=r'$r_I$')\n", " plt.xlabel('time(ms)')\n", " plt.legend()\n", " plt.ylabel('Firing Rate (Hz)')\n", " plt.show()\n", "\n", "\n", "def plot_rErI_Simple(t, r_E, r_I):\n", " \"\"\"\n", " Args:\n", " t : time\n", " r_E : excitation rate\n", " r_I : inhibition rate\n", "\n", " Returns:\n", " figure with two panels\n", " left panel: r_I and r_E as a function of time\n", " right panel: r_I as a function of r_E with Nullclines\n", "\n", " \"\"\"\n", " with plt.xkcd():\n", " fig = plt.figure(figsize=(12,4))\n", " gs = gridspec.GridSpec(1, 2)\n", " # LEFT PANEL\n", " plt.subplot(gs)\n", " plt.plot(t,r_E,':',color='b',label=r'$r_E$')\n", " plt.plot(t,r_I,':',color='r',label=r'$r_I$')\n", " plt.xlabel('time(ms)')\n", " plt.legend()\n", " plt.ylabel('Firing Rate (Hz)')\n", " # RIGHT PANEL\n", " plt.subplot(gs)\n", " plt.plot(r_E,r_I,'k:')\n", " plt.plot(r_E,r_I,'go')\n", "\n", " plt.hlines(0,np.min(r_E),np.max(r_E),linestyles=\"dashed\",color='b',label=r'$\\frac{d}{dt}r_E=0$')\n", " plt.vlines(0,np.min(r_I),np.max(r_I),linestyles=\"dashed\",color='r',label=r'$\\frac{d}{dt}r_I=0$')\n", "\n", " plt.legend(loc='upper left')\n", "\n", " plt.xlabel(r'$r_E$')\n", " plt.ylabel(r'$r_I$')\n", " plt.show()\n", "\n", "def plot_rErI_Matrix(t, r_E, r_I, Null_rE, Null_rI):\n", " \"\"\"\n", " Args:\n", " t : time\n", " r_E : excitation firing rate\n", " r_I : inhibition firing rate\n", " Null_rE: Nullclines excitation firing rate\n", " Null_rI: Nullclines inhibition firing rate\n", " Returns:\n", " figure with two panels\n", " left panel: r_I and r_E as a function of time\n", " right panel: r_I as a function of r_E with Nullclines\n", "\n", " \"\"\"\n", "\n", " with plt.xkcd():\n", " fig = plt.figure(figsize=(12,4))\n", " gs = gridspec.GridSpec(1, 2)\n", " plt.subplot(gs)\n", " plt.plot(t,r_E,':',color='b',label=r'$r_E$')\n", " plt.plot(t,r_I,':',color='r',label=r'$r_I$')\n", " plt.xlabel('time(ms)')\n", " plt.ylabel('Firing Rate (Hz)')\n", " plt.legend()\n", " plt.subplot(gs)\n", " plt.plot(r_E,r_I,'k:')\n", " plt.plot(r_E,r_I,'go')\n", "\n", " plt.plot(r_E,Null_rE,':',color='b',label=r'$\\frac{d}{dt}r_E=0$')\n", " plt.plot(r_E,Null_rI,':',color='r',label=r'$\\frac{d}{dt}r_I=0$')\n", " plt.legend(loc='best')\n", " plt.xlabel(r'$r_E$')\n", " plt.ylabel(r'$r_I$')\n", " plt.show()" ] }, { "cell_type": "markdown", "id": "SXJPvBQzik6K", "metadata": { "execution": {} }, "source": [ "---\n", "# Section 1: Intro to the Euler method using the population differential equation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Video 1: Intro to numerical methods for differential equations\n" ] }, { "cell_type": "code", "execution_count": null, "id": "DYKXWaR7iwl9", "metadata": { "cellView": "form", "execution": {}, "tags": [ "remove-input" ] }, "outputs": [], "source": [ "# @title Video 1: Intro to numerical methods for differential equations\n", "from ipywidgets import widgets\n", "from IPython.display import YouTubeVideo\n", "from IPython.display import IFrame\n", "from IPython.display import display\n", "\n", "\n", "class PlayVideo(IFrame):\n", " def __init__(self, id, source, page=1, width=400, height=300, **kwargs):\n", " self.id = id\n", " if source == 'Bilibili':\n", " src = f'https://player.bilibili.com/player.html?bvid={id}&page={page}'\n", " elif source == 'Osf':\n", " src = f'https://mfr.ca-1.osf.io/render?url=https://osf.io/download/{id}/?direct%26mode=render'\n", " super(PlayVideo, self).__init__(src, width, height, **kwargs)\n", "\n", "\n", "def display_videos(video_ids, W=400, H=300, fs=1):\n", " tab_contents = []\n", " for i, video_id in enumerate(video_ids):\n", " out = widgets.Output()\n", " with out:\n", " if video_ids[i] == 'Youtube':\n", " video = YouTubeVideo(id=video_ids[i], width=W,\n", " height=H, fs=fs, rel=0)\n", " print(f'Video available at https://youtube.com/watch?v={video.id}')\n", " else:\n", " video = PlayVideo(id=video_ids[i], source=video_ids[i], width=W,\n", " height=H, fs=fs, autoplay=False)\n", " if video_ids[i] == 'Bilibili':\n", " print(f'Video available at https://www.bilibili.com/video/{video.id}')\n", " elif video_ids[i] == 'Osf':\n", " print(f'Video available at https://osf.io/{video.id}')\n", " display(video)\n", " tab_contents.append(out)\n", " return tab_contents\n", "\n", "\n", "video_ids = [('Youtube', 'NI8c80TA7IQ'), ('Bilibili', 'BV1gh411Y7gV')]\n", "tab_contents = display_videos(video_ids, W=730, H=410)\n", "tabs = widgets.Tab()\n", "tabs.children = tab_contents\n", "for i in range(len(tab_contents)):\n", " tabs.set_title(i, video_ids[i])\n", "display(tabs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "M3V5Kju3j7Ie", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Intro_to_numerical_methods_for_differential_equations_Video\")" ] }, { "cell_type": "markdown", "id": "R9B9JKT-PvEn", "metadata": { "execution": {} }, "source": [ "## Section 1.1: Slope of line as approximation of derivative\n", "\n", "*Estimated timing to here from start of tutorial: 8 min*\n", "\n", "
\n", " Click here for text recap of relevant part of video \n", "\n", "The Euler method is one of the straight forward and elegant methods to approximate a differential. It was designed by [Leonhard Euler](https://en.wikipedia.org/wiki/Leonhard_Euler) (1707-1783).\n", "Simply put we just replace the derivative in the differential equation by the formula for a line and re-arrange.\n", "\n", "The slope is the rate of change between two points. The formula for the slope of a line between the points $(t_0,y(t_0))$ and $(t_1,y(t_1))$ is given by:\n", "$$m=\\frac{y(t_1)-y(t_0)}{t_1-t_0},$$\n", "which can be written as\n", "$$m=\\frac{y_1-y_0}{t_1-t_0},$$\n", "or as\n", "$$m=\\frac{\\Delta y_0}{\\Delta t_0},$$\n", "where $\\Delta y_0=y_1-y_0$ and $\\Delta t_0=t_1-t_0$ or in words as\n", "$$m=\\frac{\\text{ Change in y} }{\\text{Change in t}}.$$\n", "The slope can be used as an approximation of the derivative such that\n", "$$\\frac{d}{dt}y(t)\\approx \\frac{y(t_0+\\Delta t)-y(t_0)}{t_0+\\Delta t-t_0}=\\frac{y(t_0+dt)-y(t_0)}{\\Delta t}$$\n", "where $\\Delta t$ is a time-step.\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "b667c36a", "metadata": { "execution": {} }, "source": [ "### Interactive Demo 1.1: Slope of a Line\n", "\n", "The plot below shows a function $y(t)$ in blue, its exact derivative $\\frac{d}{dt}y(t)$ at $t_0=1$ in black. The approximate derivative is calculated using the slope formula $=\\frac{y(1+\\Delta t)-y(1)}{\\Delta t}$ for different time-steps sizes $\\Delta t$ in green.\n", "\n", "Interact with the widget to see how time-step impacts the slope's accuracy, which is the derivative's estimate.\n", "- How does the size of $\\Delta t$ affect the approximation?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Make sure you execute this cell to enable the widget!\n" ] }, { "cell_type": "code", "execution_count": null, "id": "ea4a4f78", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown Make sure you execute this cell to enable the widget!\n", "my_layout.width = '450px'\n", "@widgets.interact(\n", "\n", " dt=widgets.FloatSlider(1, min=0., max=4., step=.1,\n", " layout=my_layout)\n", "\n", ")\n", "\n", "def Pop_widget(dt):\n", " plot_slope(dt)\n", " plt.show()" ] }, { "cell_type": "markdown", "id": "gTgFRiMhc1qm", "metadata": { "colab_type": "text", "execution": {} }, "source": [ "[*Click for solution*](https://github.com/NeuromatchAcademy/precourse/tree/main/tutorials/W0D4_Calculus/solutions/W0D4_Tutorial3_Solution_ad3e347b.py)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "FZ1SZPQwj9H4", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Slope_of_a_line_Interactive_Demo\")" ] }, { "cell_type": "markdown", "id": "8f92ce83", "metadata": { "execution": {} }, "source": [ "## Section 1.2: Euler error for a single step\n", "\n", "*Estimated timing to here from start of tutorial: 12 min*\n", "\n", "
\n", " Click here for text recap of relevant part of video \n", "\n", "Linking with the previous tutorial, we will use the population differential equation to get an intuitive feel of using the Euler method to approximate the solution of a differential equation, as it has an exact solution with no discontinuities.\n", "\n", "The population differential equation is given by\n", "\\begin{align*}\n", "\\\\\n", "\\frac{d}{dt}\\,p(t) &= \\alpha p(t)\\\\\\\\\n", "p(0)&=p_0 \\quad \\text{Initial Condition}\n", "\\end{align*}\n", "\n", "where $p(t)$ is the population at time $t$ and $\\alpha$ is a parameter representing birth rate. The exact solution of the differential equation is\n", "$$p(t)=p_0e^{\\alpha t}.$$\n", "\n", "To numerically estimate the population differential equation we replace the derivate with the slope of the line to get the discrete (not continuous) equation:\n", "\n", "\\begin{align*}\n", "\\\\\n", "\\frac{p_1-p_0}{t_1-t_0} &= \\alpha p_0\\\\\\\\\n", "p(0)&=p_0 \\quad \\text{Initial Condition}\n", "\\end{align*}\n", "\n", "where $p_1$ is the estimate of $p(t_1)$. Let $\\Delta t=t_1-t_0$ be the time-step and re-arrange the equation gives\n", "\n", "\\begin{align*}\n", "\\color{red}{p_1}&=\\color{green}{p_0}+\\color{blue}{\\Delta t} (\\color{blue}{\\alpha} \\color{green}{p_0})\n", "\\end{align*}\n", "\n", "where $\\color{red}{p_1}$ is the unknown future, $\\color{green}{p_0}$ is the known current population, $\\color{blue}{\\Delta t}$ is the chosen time-step parameter and $\\color{blue}{\\alpha}$ is the given birth rate parameter.\n", "Another way to read the re-arranged discrete equation is:\n", "\n", "\\begin{align*}\n", "\\color{red}{\\text{\"Future Population\"}}&=\\color{green}{\\text{ \"Current Population\"}}+\\color{blue}{\\text{ time-step}} \\times (\\color{blue}{\\text{ \"Birth rate}}\\times \\color{green}{\\text{ Current Population\"}}).\n", "\\end{align*}\n", "\n", "So pretty much, we can use the current population and add a bit of the dynamics of the differential equation to predict the future. We will make millions... But wait there is always an error.\n", "\n", "The solution of the Euler method $p_1$ is an estimate of the exact solution $p(t_1)$ at $t_1$ which means there is a bit of error $e_1$ which gives the equation\n", "\\begin{align*}\n", "p(t_1)&=p_1+e_1\\\\\\\\\n", "\\text{Rearranging}\\\\\\\\\n", "e_1&=p(t_1)-p_1,\\\\\n", "\\text{Error}&=\\text{Exact-Estimate}.\n", "\\end{align*}\n", "\n", "Most of the time we do not know the exact answer $p(t_1)$ and hence the size of the error $e_1$ but for the population equation we have the exact solution $p(t)=p_0e^{\\alpha}.$\n", "\n", "This means we can explore what the error looks like.\n", "
" ] }, { "cell_type": "markdown", "id": "vGxg-2k6k8hM", "metadata": { "execution": {} }, "source": [ "### Interactive Demo 1.2: Euler error for a single step\n", "\n", "Interact with the widget to see how the time-step $\\Delta t$, dt in code, impacts the estimate $p_1$ and the error $e_1$ of the Euler method.\n", "\n", "1. What happens to the estimate $p_1$ as the time-step $\\Delta t$ increases?\n", "2. Is there a relationship between the size of $\\Delta t$ and $e_1$?\n", "3. How would you improve the error $e_1$?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Make sure you execute this cell to enable the widget!\n" ] }, { "cell_type": "code", "execution_count": null, "id": "e72c600b", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown Make sure you execute this cell to enable the widget!\n", "my_layout.width = '450px'\n", "@widgets.interact(\n", " dt=widgets.FloatSlider(1.5, min=0., max=4., step=.1,\n", " layout=my_layout)\n", ")\n", "\n", "def Pop_widget(dt):\n", " plot_StepEuler(dt)\n", " plt.show()" ] }, { "cell_type": "markdown", "id": "EoA90TrDdVWv", "metadata": { "colab_type": "text", "execution": {} }, "source": [ "[*Click for solution*](https://github.com/NeuromatchAcademy/precourse/tree/main/tutorials/W0D4_Calculus/solutions/W0D4_Tutorial3_Solution_9829732b.py)\n", "\n" ] }, { "cell_type": "markdown", "id": "e78fc157", "metadata": { "execution": {} }, "source": [ "The error $e_1$ from one time-step is known as the __local error__. For the Euler method, the local error is linear, which means that the error decreases linearly as a function of timestep and is written as $\\mathcal{O}(\\Delta t)$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "qXS6Fkkkj-xD", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Euler_error_of_single_step_Interactive_Demo\")" ] }, { "cell_type": "markdown", "id": "1yr4hKV7R0JJ", "metadata": { "execution": {} }, "source": [ "## Section 1.3: Taking more steps\n", "\n", "*Estimated timing to here from start of tutorial: 16 min*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Video 2: Taking more steps\n" ] }, { "cell_type": "code", "execution_count": null, "id": "MZcqv_DoRsWJ", "metadata": { "cellView": "form", "execution": {}, "tags": [ "remove-input" ] }, "outputs": [], "source": [ "# @title Video 2: Taking more steps\n", "from ipywidgets import widgets\n", "from IPython.display import YouTubeVideo\n", "from IPython.display import IFrame\n", "from IPython.display import display\n", "\n", "\n", "class PlayVideo(IFrame):\n", " def __init__(self, id, source, page=1, width=400, height=300, **kwargs):\n", " self.id = id\n", " if source == 'Bilibili':\n", " src = f'https://player.bilibili.com/player.html?bvid={id}&page={page}'\n", " elif source == 'Osf':\n", " src = f'https://mfr.ca-1.osf.io/render?url=https://osf.io/download/{id}/?direct%26mode=render'\n", " super(PlayVideo, self).__init__(src, width, height, **kwargs)\n", "\n", "\n", "def display_videos(video_ids, W=400, H=300, fs=1):\n", " tab_contents = []\n", " for i, video_id in enumerate(video_ids):\n", " out = widgets.Output()\n", " with out:\n", " if video_ids[i] == 'Youtube':\n", " video = YouTubeVideo(id=video_ids[i], width=W,\n", " height=H, fs=fs, rel=0)\n", " print(f'Video available at https://youtube.com/watch?v={video.id}')\n", " else:\n", " video = PlayVideo(id=video_ids[i], source=video_ids[i], width=W,\n", " height=H, fs=fs, autoplay=False)\n", " if video_ids[i] == 'Bilibili':\n", " print(f'Video available at https://www.bilibili.com/video/{video.id}')\n", " elif video_ids[i] == 'Osf':\n", " print(f'Video available at https://osf.io/{video.id}')\n", " display(video)\n", " tab_contents.append(out)\n", " return tab_contents\n", "\n", "\n", "video_ids = [('Youtube', 'cGsXHllGMVo'), ('Bilibili', 'BV135411T7QF')]\n", "tab_contents = display_videos(video_ids, W=730, H=410)\n", "tabs = widgets.Tab()\n", "tabs.children = tab_contents\n", "for i in range(len(tab_contents)):\n", " tabs.set_title(i, video_ids[i])\n", "display(tabs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "W8HbORAlkAWj", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Taking_more_steps_Video\")" ] }, { "cell_type": "markdown", "id": "972a55c4", "metadata": { "execution": {} }, "source": [ "
\n", " Click here for text recap of relevant part of video \n", "\n", "In the above exercise we saw that by increasing the time-step $\\Delta t$ the error between the estimate $p_1$ and the exact $p(t_1)$ increased. The largest error in the example above was for $\\Delta t=4$, meaning the first time point was at 1 year and the second was at 5 years (as 5 - 1 = 4).\n", "\n", "To decrease the error, we can divide the interval $[1, 5]$ into more steps using a smaller $\\Delta t$.\n", "\n", "In the plot below we use $\\Delta t=1$, which divides the interval into four segments\n", "$$n=\\frac{5-1}{1}=4,$$\n", "giving\n", "$$t_0=t=1, \\ t_1=t=2, \\ t_2=t=3, \\ t_3=t=4 \\ \\text{ and } t_4=t=5.$$\n", "This can be written as\n", "$$t[k]=1+k\\times1, \\text{ for } k=0,\\cdots 4,$$\n", "and more generally as\n", "$$t[k]=t[k]+k\\times \\Delta t, \\text{ for } k=0,\\cdots n,$$\n", "where $n$ is the number of steps.\n", "\n", "Using the Euler method, the continuous population differential equation is written as a series of discrete difference equations of the form:\n", "\\begin{align*}\n", "\\color{red}{p[k+1]}&=\\color{green}{p[k]}+\\color{blue}{\\Delta t} (\\color{blue}{\\alpha} \\color{green}{p[k]})\\\\\n", "&\\text{for } k=0,1,\\cdots n-1\n", "\\end{align*}\n", "where $\\color{red}{p[k+1]}$ is the unknown estimate of the future population at $t[k+1]$, $\\color{green}{p[k]}$ is the known current population estimate at $t_k$, $\\color{blue}{\\Delta t}$ is the chosen time-step parameter and $\\color{blue}{\\alpha}$ is the given birth rate parameter.\n", "\n", "\n", "The Euler method can be applied to all first order differential equations of the form\n", "\\begin{align*}\n", "\\frac{d}{dt}y(t)&=f(t,y(t)),\\\\\n", "y(t_{0})&=y_0,\\\\\n", "\\end{align*}\n", "on an interval $[a,b]$.\n", "\n", "Using the Euler method all differential equation can be written as discrete difference equations:\n", "\\begin{align*}\n", "\\frac{\\color{red}{y[k+1]}-\\color{green}{y[k]}}{\\color{blue}{\\Delta t}}&=f(\\color{blue}{t[k]},\\color{green}{y[k]}),\\\\\n", "\\text{Re-arranging}\\\\\n", "\\color{red}{y[k+1]}&=\\color{green}{y[k]}+\\color{blue}{\\Delta t}f(\\color{blue}{t[k]},\\color{green}{y[k]}),\\\\\n", "&\\text{ for } k=0, \\cdots n-1,\\\\\n", "y&=\\color{green}{y_0},\\\\\n", "\\end{align*}\n", "where $\\color{red}{y[k+1]}$ is the unknown estimate at $t[k+1]$, $\\color{green}{y[k]}$ is the known at $t_k$, $\\color{blue}{\\Delta t}$ is the chosen time-step parameter, $\\color{blue}{t[k]}$ is the time point and $f$ is the right hand side of the differential equation.\n", "The discrete time steps are:\n", "\\begin{align*}\n", "\\color{blue}{t[k]}&=\\color{blue}{t}+\\color{blue}{k}\\color{blue}{\\Delta t},\\\\\n", "n&=\\frac{b-a}{\\Delta t}\\\\\n", "&\\text{ for } k=0, \\cdots n.\\\\\n", "\\end{align*}\n", "Once again this can be simply put into words:\n", "\\begin{align*}\n", "\\color{red}{\\text{ \"Future\" }}&=\\color{green}{\\text{ \"Current Info\" }}+\\color{blue}{\\text{ time-step } }\\times\\text{ \"Dynamics of the system which is a function of } \\color{blue}{ \\text{ time }} \\text{ and }\\color{green}{ \\text{ Current Info.\" }}\n", "\\end{align*}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " *Execute this cell to visualize time steps*\n" ] }, { "cell_type": "code", "execution_count": null, "id": "d84f02ed", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown *Execute this cell to visualize time steps*\n", "dt =1\n", "t =np.arange(1, 5+dt/2, dt) # Time from 1 to 5 years in 0.1 steps\n", "with plt.xkcd():\n", " fig = plt.figure(figsize=(8, 2))\n", " plt.plot(t, 0*t, ':o')\n", " plt.plot(t ,0*t,':go',label='Initial Condition')\n", " plt.text(t-0.1, 0*t-0.03, r'$t_0$')\n", " plt.text(t-0.1, 0*t-0.03, r'$t_1$')\n", " plt.text(t-0.1, 0*t-0.03, r'$t_2$')\n", " plt.text(t-0.1, 0*t-0.03, r'$t_3$')\n", " plt.text(t-0.1, 0*t-0.03,r'$t_4$')\n", " plt.text(t+0.5, 0*t+0.02, r'$\\Delta t$')\n", " plt.text(t+0.5, 0*t+0.02, r'$\\Delta t$')\n", " plt.text(t+0.5, 0*t+0.02, r'$\\Delta t$')\n", " plt.text(t+0.5, 0*t+0.02, r'$\\Delta t$')\n", " plt.yticks([])#plt.ylabel('Population (millions)')\n", " plt.xlabel('time(years)')\n", " plt.show()" ] }, { "cell_type": "markdown", "id": "279ea532", "metadata": { "execution": {} }, "source": [ "### Coding Exercise 1.3: Step, step, step\n", "\n", "Given the population differential equation:\n", "\\begin{align*}\n", "\\frac{d}{dt}\\,p(t) &= 0.3 p(t),\\\\\n", "\\end{align*}\n", "\n", "and the initial condition:\n", "\n", "\\begin{align*}\n", "p(t_0=1)&=e^{0.3},\n", "\\end{align*}\n", "\n", "code the difference equation:\n", "\n", "\\begin{align*}\n", "\\color{red}{p[k+1]}&=\\color{green}{p[k]}+\\color{blue}{\\Delta t} (\\color{blue}{0.3} \\color{green}{p[k]}),\\\\\n", "\\color{green}{p}&=e^{0.3},\\quad \\text{Initial Condition,}\\\\\n", "&\\text{for } k=0,1,\\cdots 4,\\\\\n", "\\end{align*}\n", "\n", "to estimate the population on the interval $[1,5]$ with a time-step $\\Delta t=1$, denoted by dt in code." ] }, { "cell_type": "code", "execution_count": null, "id": "8b872e66", "metadata": { "execution": {} }, "outputs": [], "source": [ "# Time step\n", "dt = 1\n", "\n", "# Make time range from 1 to 5 years with step size dt\n", "t = np.arange(1, 5+dt/2, dt)\n", "\n", "# Get number of steps\n", "n = len(t)\n", "\n", "# Initialize p array\n", "p = np.zeros(n)\n", "p = np.exp(0.3*t) # initial condition\n", "\n", "# Loop over steps\n", "for k in range(n-1):\n", "\n", " ########################################################################\n", " ## TODO for students\n", " ## Complete line of code and remove\n", " raise NotImplementedError(\"Student exercise: calculate the population step for each time point\")\n", " ########################################################################\n", "\n", " # Calculate the population step\n", " p[k+1] = ...\n", "\n", "# Visualize\n", "visualize_population_approx(t, p)" ] }, { "cell_type": "markdown", "id": "LHAKHn_5UzLQ", "metadata": { "colab_type": "text", "execution": {} }, "source": [ "[*Click for solution*](https://github.com/NeuromatchAcademy/precourse/tree/main/tutorials/W0D4_Calculus/solutions/W0D4_Tutorial3_Solution_dffaca6b.py)\n", "\n", "*Example output:*\n", "\n", " \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "Yvl3OBBPkCQr", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Step_step_step_Exercise\")" ] }, { "cell_type": "markdown", "id": "d9301447", "metadata": { "execution": {} }, "source": [ "The error is smaller for 4-time steps than taking one large time step from 1 to 5 but does note that the error is increasing for each step. This is known as __global error__ so the further in time you want to predict, the larger the error.\n", "\n", "You can read the theorems [here](https://colab.research.google.com/github/john-s-butler-dit/Numerical-Analysis-Python/blob/master/Chapter%2001%20-%20Euler%20Methods/102_Euler_method_with_Theorems_nonlinear_Growth_function.ipynb)." ] }, { "cell_type": "markdown", "id": "b419bf50", "metadata": { "execution": {} }, "source": [ "---\n", "# Section 2: Euler method for the leaky integrate and fire\n", "\n", "\n", "*Estimated timing to here from start of tutorial: 26 min*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Video 3: Leaky integrate and fire\n" ] }, { "cell_type": "code", "execution_count": null, "id": "JOA7EzauxIV6", "metadata": { "cellView": "form", "execution": {}, "tags": [ "remove-input" ] }, "outputs": [], "source": [ "# @title Video 3: Leaky integrate and fire\n", "from ipywidgets import widgets\n", "from IPython.display import YouTubeVideo\n", "from IPython.display import IFrame\n", "from IPython.display import display\n", "\n", "\n", "class PlayVideo(IFrame):\n", " def __init__(self, id, source, page=1, width=400, height=300, **kwargs):\n", " self.id = id\n", " if source == 'Bilibili':\n", " src = f'https://player.bilibili.com/player.html?bvid={id}&page={page}'\n", " elif source == 'Osf':\n", " src = f'https://mfr.ca-1.osf.io/render?url=https://osf.io/download/{id}/?direct%26mode=render'\n", " super(PlayVideo, self).__init__(src, width, height, **kwargs)\n", "\n", "\n", "def display_videos(video_ids, W=400, H=300, fs=1):\n", " tab_contents = []\n", " for i, video_id in enumerate(video_ids):\n", " out = widgets.Output()\n", " with out:\n", " if video_ids[i] == 'Youtube':\n", " video = YouTubeVideo(id=video_ids[i], width=W,\n", " height=H, fs=fs, rel=0)\n", " print(f'Video available at https://youtube.com/watch?v={video.id}')\n", " else:\n", " video = PlayVideo(id=video_ids[i], source=video_ids[i], width=W,\n", " height=H, fs=fs, autoplay=False)\n", " if video_ids[i] == 'Bilibili':\n", " print(f'Video available at https://www.bilibili.com/video/{video.id}')\n", " elif video_ids[i] == 'Osf':\n", " print(f'Video available at https://osf.io/{video.id}')\n", " display(video)\n", " tab_contents.append(out)\n", " return tab_contents\n", "\n", "\n", "video_ids = [('Youtube', 'T85OcgY7xjo'), ('Bilibili', 'BV1k5411T7by')]\n", "tab_contents = display_videos(video_ids, W=730, H=410)\n", "tabs = widgets.Tab()\n", "tabs.children = tab_contents\n", "for i in range(len(tab_contents)):\n", " tabs.set_title(i, video_ids[i])\n", "display(tabs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "GeAbFXiSkDEd", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Leaky_integrate_and_fire_Video\")" ] }, { "cell_type": "markdown", "id": "Zjro4hXwxU8O", "metadata": { "execution": {} }, "source": [ "
\n", " Click here for text recap of video \n", "\n", "The Leaky Integrate and Fire (LIF) differential equation is:\n", "\n", "\\begin{align}\n", "\\frac{dV}{dt} = \\frac{-(V-E_L) + R_mI(t)}{\\tau_m}\\,\n", "\\end{align}\n", "\n", "where $\\tau_m$ is the time constant, $V$ is the membrane potential, $E_L$ is the resting potential, $R_m$ is leak resistance and $I(t)$ is the external input current.\n", "\n", "The solution of the LIF can be estimated by applying the Euler method to give the difference equation:\n", "\n", "\\begin{align}\n", "\\frac{\\color{red}{V[k+1]}-\\color{green}{V[k]}}{\\color{blue}{\\Delta t}}=\\big(\\frac{-(\\color{green}{V[k]}-\\color{blue}{E_L}) + \\color{blue}{R_m}I[k]}{\\color{blue}{\\tau_m}}\\big),\\\\\n", "\\end{align}\n", "where $V[k]$ is the estimate of the membrane potential at time point $t[k]$.\n", "Re-arranging the equation such that all the known terms are on the right gives:\n", "\n", "\\begin{align}\n", "\\color{red}{V[k+1]}=\\color{green}{V[k]}+\\color{blue}{\\Delta t}\\big(\\frac{-(\\color{green}{V[k]}-\\color{blue}{E_L}) + \\color{blue}{R_m}I[k]}{\\color{blue}{\\tau_m}}\\big),\\\\\n", "\\text{for } k=0\\cdots n-1,\n", "\\end{align}\n", "\n", "where $\\color{red}{V[k+1]}$ is the unknown membrane potential at $t[k+1]$, $\\color{green}{V[k]}$ is known membrane potential, $\\color{blue}{E_L}$, $\\color{blue}{R_m}$ and $\\color{blue}{\\tau_m}$ are known parameters, $\\color{blue}{\\Delta t}$ is a chosen time-step and $I(t_k)$ is a function for an external input current." ] }, { "cell_type": "markdown", "id": "1efb9d89", "metadata": { "execution": {} }, "source": [ "## Coding Exercise 2: LIF and Euler\n", "Code the difference equation for the LIF:\n", "\n", "\\begin{align}\n", "\\color{red}{V[k+1]}=\\color{green}{V[k]}+\\color{blue}{\\Delta t}\\big(\\frac{-(\\color{green}{V[k]}-\\color{blue}{E_L}) + \\color{blue}{R_m}I[k]}{\\color{blue}{\\tau_m}}\\big),\\\\\n", "\\text{for } k=0\\cdots n-1,\n", "\\end{align}\n", "\n", "with the given parameters set as:\n", "* V_reset = -75,\n", "* E_L = -75,\n", "* tau_m = 10,\n", "* R_m = 10.\n", "\n", "We will then visualize the result.\n", "The figure has three panels:\n", "* the top panel is the sinusoidal input, $I$,\n", "* the middle panel is the estimate membrane potential $V_k$. To illustrate a spike, $V_k$ is set to $0$ and then reset,\n", "* the bottom panel is the raster plot with each dot indicating a spike." ] }, { "cell_type": "code", "execution_count": null, "id": "4e7d29bb", "metadata": { "execution": {} }, "outputs": [], "source": [ "def Euler_Integrate_and_Fire(I, time, dt):\n", " \"\"\"\n", " Args:\n", " I: Input\n", " time: time\n", " dt: time-step\n", " Returns:\n", " Spike: Spike count\n", " Spike_time: Spike times\n", " V: membrane potential esitmated by the Euler method\n", " \"\"\"\n", "\n", " Spike = 0\n", " tau_m = 10\n", " R_m = 10\n", " t_isi = 0\n", " V_reset = E_L = -75\n", " n = len(time)\n", " V = V_reset * np.ones(n)\n", " V_th = -50\n", " Spike_time = []\n", "\n", " for k in range(n-1):\n", " #######################################################################\n", " ## TODO for students: calculate the estimate solution of V at t[i+1]\n", " ## Complete line of codes for dV and remove\n", " ## Run the code in Section 5.1 or 5.2 to see the output!\n", " raise NotImplementedError(\"Student exercise: calculate the estimate solution of V at t[i+1]\")\n", " ########################################################################\n", "\n", " dV = ...\n", " V[k+1] = V[k] + dt*dV\n", "\n", " # Discontinuity for Spike\n", " if V[k] > V_th:\n", " V[k] = 0\n", " V[k+1] = V_reset\n", " t_isi = time[k]\n", " Spike = Spike + 1\n", " Spike_time = np.append(Spike_time, time[k])\n", "\n", " return Spike, Spike_time, V\n", "\n", "# Set up time step and current\n", "dt = 1\n", "t = np.arange(0, 1000, dt)\n", "I = np.sin(4 * 2 * np.pi * t/1000) + 2\n", "\n", "# Model integrate and fire neuron\n", "Spike, Spike_time, V = Euler_Integrate_and_Fire(I, t, dt)\n", "\n", "# Visualize\n", "plot_IF(t, V,I,Spike_time)" ] }, { "cell_type": "markdown", "id": "55785e26", "metadata": { "colab_type": "text", "execution": {} }, "source": [ "[*Click for solution*](https://github.com/NeuromatchAcademy/precourse/tree/main/tutorials/W0D4_Calculus/solutions/W0D4_Tutorial3_Solution_874eaa29.py)\n", "\n", "*Example output:*\n", "\n", " \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "ggAQjBCekEm7", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_LIF_and_Euler_Exercise\")" ] }, { "cell_type": "markdown", "id": "0d322dc9", "metadata": { "execution": {} }, "source": [ "As electrophysiologists normally listen to spikes when conducting experiments, listen to the music of the LIF neuron below. Note: this does not work on all browsers so just move on if you can't hear the audio." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " *Execute this cell to visualize the LIF for sinusoidal input*\n" ] }, { "cell_type": "code", "execution_count": null, "id": "cd8f60c2", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown *Execute this cell to visualize the LIF for sinusoidal input*\n", "\n", "dt = 1\n", "t = np.arange(0, 1000, dt)\n", "I = np.sin(4 * 2 * np.pi * t/1000) + 2\n", "Spike, Spike_time, V = Euler_Integrate_and_Fire(I, t, dt)\n", "\n", "plot_IF(t, V,I,Spike_time)\n", "plt.show()\n", "ipd.Audio(V, rate=1000/dt)" ] }, { "cell_type": "markdown", "id": "L5o2liqz3bxi", "metadata": { "execution": {} }, "source": [ "---\n", "# Section 3: Systems of differential equations\n", "\n", "\n", "*Estimated timing to here from start of tutorial: 34 min*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Video 4: Systems of differential equations\n" ] }, { "cell_type": "code", "execution_count": null, "id": "RkM3-RucX2c-", "metadata": { "cellView": "form", "execution": {}, "tags": [ "remove-input" ] }, "outputs": [], "source": [ "# @title Video 4: Systems of differential equations\n", "from ipywidgets import widgets\n", "from IPython.display import YouTubeVideo\n", "from IPython.display import IFrame\n", "from IPython.display import display\n", "\n", "\n", "class PlayVideo(IFrame):\n", " def __init__(self, id, source, page=1, width=400, height=300, **kwargs):\n", " self.id = id\n", " if source == 'Bilibili':\n", " src = f'https://player.bilibili.com/player.html?bvid={id}&page={page}'\n", " elif source == 'Osf':\n", " src = f'https://mfr.ca-1.osf.io/render?url=https://osf.io/download/{id}/?direct%26mode=render'\n", " super(PlayVideo, self).__init__(src, width, height, **kwargs)\n", "\n", "\n", "def display_videos(video_ids, W=400, H=300, fs=1):\n", " tab_contents = []\n", " for i, video_id in enumerate(video_ids):\n", " out = widgets.Output()\n", " with out:\n", " if video_ids[i] == 'Youtube':\n", " video = YouTubeVideo(id=video_ids[i], width=W,\n", " height=H, fs=fs, rel=0)\n", " print(f'Video available at https://youtube.com/watch?v={video.id}')\n", " else:\n", " video = PlayVideo(id=video_ids[i], source=video_ids[i], width=W,\n", " height=H, fs=fs, autoplay=False)\n", " if video_ids[i] == 'Bilibili':\n", " print(f'Video available at https://www.bilibili.com/video/{video.id}')\n", " elif video_ids[i] == 'Osf':\n", " print(f'Video available at https://osf.io/{video.id}')\n", " display(video)\n", " tab_contents.append(out)\n", " return tab_contents\n", "\n", "\n", "video_ids = [('Youtube', 'A3Puozl9nEs'), ('Bilibili', 'BV1XV411s76a')]\n", "tab_contents = display_videos(video_ids, W=730, H=410)\n", "tabs = widgets.Tab()\n", "tabs.children = tab_contents\n", "for i in range(len(tab_contents)):\n", " tabs.set_title(i, video_ids[i])\n", "display(tabs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "x5wo7eFJkGBS", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Systems_of_differential_equations_Video\")" ] }, { "cell_type": "markdown", "id": "dTy23IBPZxYG", "metadata": { "execution": {} }, "source": [ "## Section 3.1: Using Euler to approximate a simple system\n", "\n", "*Estimated timing to here from start of tutorial: 40 min*" ] }, { "cell_type": "markdown", "id": "85df22b9", "metadata": { "execution": {} }, "source": [ "
\n", " Click here for text recap of relevant part of video \n", "\n", "To get to grips with solving a system of differential equations using the Euler method, we will simplify the Wilson Cowan model, a set of equations that will be explored in more detail in the Dynamical Systems day.\n", "Looking at systems of differential equations will also allow us to introduce the concept of a phase-plane plot which is a method of investigating how different processes interact.\n", "\n", "In the previous section we looked at the LIF model for a single neuron. We now model a collection of neurons using a differential equation which describes the firing rate of a population of neurons.\n", "We will model the firing rate $r$ of two types of populations of neurons which interact, the excitation population firing rate $r_E$ and inhibition population firing rate $r_I$. These firing rates of neurons regulate each other by weighted connections $w$. The directed graph below illustrates this.\n", "\n", "Our system of differential equations is a linear version of the Wilson Cowan model. Consider the equations,\n", "\n", "\\begin{align}\n", "\\tau_E \\frac{dr_E}{dt} &= w_{EE}r_E+w_{EI}r_I, \\\\\n", "\\tau_I \\frac{dr_I}{dt} &= w_{IE}r_E+w_{II}r_I\n", "\\end{align}\n", "\n", "$r_E(t)$ represents the average activation (or firing rate) of the excitatory population at time $t$, and $r_I(t)$ the activation (or firing rate) of the inhibitory population. The parameters $\\tau_E$ and $\\tau_I$ control the timescales of the dynamics of each population. Connection strengths are given by: $w_{EE}$ (E $\\rightarrow$ E), $w_{EI}$ (I $\\rightarrow$ E), $w_{IE}$ (E $\\rightarrow$ I), and $w_{II}$ (I $\\rightarrow$ I). The terms $w_{EI}$ and $w_{IE}$ represent connections from inhibitory to excitatory population and vice versa, respectively.\n", "\n", "To start we will further simplify the linear system of equations by setting $w_{EE}$ and $w_{II}$ to zero, we now have the equations\n", "\n", "\\begin{align}\n", "\\tau_E \\frac{dr_E}{dt} &= w_{EI}r_I, \\\\\n", "\\tau_I \\frac{dr_I}{dt} &= w_{IE}r_E, \\qquad (1)\n", "\\end{align}\n", "\n", "where $\\tau_E=100$ and $\\tau_I=120$, no internal connection $w_{EE}=w_{II}=0$, and $w_{EI}=-1$ and $w_{IE}=1$,\n", "with the initial conditions\n", "\n", "\\begin{align}\n", "r_E(0)=30, \\\\\n", "r_I(0)=20.\n", "\\end{align}\n", "\n", "The solution can be approximated using the Euler method such that we have the difference equations:\n", "\n", "\\begin{align*}\n", "\\frac{\\color{red}{r_E[k+1]}-\\color{green}{r_E[k]}}{\\color{blue}{\\Delta t}}&=\\big(\\frac{\\color{blue}{w_{EI}}\\color{green}{r_I[k]}}{\\color{blue}{\\tau_E}}\\big),\\\\\n", "\\frac{\\color{red}{r_I[k+1]}-\\color{green}{r_I[k]}}{\\color{blue}{\\Delta t}}&=\\big(\\frac{\\color{blue}{w_{IE}}\\color{green}{r_E[k]}}{\\color{blue}{\\tau_I}}\\big),\\\\\n", "\\end{align*}\n", "where $r_E[k]$ and $r_I[k]$ are the numerical estimates of the firing rate of the excitation population $r_E(t[k])$ and inhibition population $r_I(t[k])$ and $\\Delta t$ is the time-step.\n", "\n", "Re-arranging the equation such that all the known terms are on the right gives:\n", "\n", "\\begin{align*}\n", "\\color{red}{r_E[k+1]}&=\\color{green}{r_E[k]}+\\color{blue}{\\Delta t}\\big(\\frac{\\color{blue}{w_{EI}}\\color{green}{r_I[k]}}{\\color{blue}{\\tau_E}}\\big),\\\\\n", "\\color{red}{r_I[k+1]}&=\\color{green}{r_I[k]}+\\color{blue}{\\Delta t}\\big(\\frac{\\color{blue}{w_{IE}}\\color{green}{r_E[k]}}{\\color{blue}{\\tau_I}}\\big),\\\\\n", "&\\text{ for } k=0, \\cdots , n-1,\\\\\\\\\n", "r_E&=30,\\\\\n", "r_I&=20.\\\\\n", "\\end{align*}\n", "\n", "where $\\color{red}{r_E[k+1]}$ and $\\color{red}{r_I[k+1]}$ are unknown, $\\color{green}{r_E[k]}$ and $\\color{green}{r_I[k]}$ are known, $\\color{blue}{w_{EI}}=-1$, $\\color{blue}{w_{IE}}=1$ and $\\color{blue}{\\tau_E}=100ms$ and $\\color{blue}{\\tau_I}=120ms$ are known parameters and $\\color{blue}{\\Delta t}=0.01ms$ is a chosen time-step.\n", "
\n", "\n" ] }, { "cell_type": "markdown", "id": "0xhGLDhgdq1P", "metadata": { "execution": {} }, "source": [ "" ] }, { "cell_type": "markdown", "id": "067b5f6e", "metadata": { "execution": {} }, "source": [ "### Coding Exercise 3.1: Euler on a Simple System\n", "\n", "Our difference equations are:\n", "\n", "\\begin{align*}\n", "\\color{red}{r_E[k+1]}&=\\color{green}{r_E[k]}+\\color{blue}{\\Delta t}\\big(\\frac{\\color{blue}{w_{EI}}\\color{green}{r_I[k]}}{\\color{blue}{\\tau_E}}\\big),\\\\\n", "\\color{red}{r_I[k+1]}&=\\color{green}{r_I[k]}+\\color{blue}{\\Delta t}\\big(\\frac{\\color{blue}{w_{IE}}\\color{green}{r_E[k]}}{\\color{blue}{\\tau_I}}\\big),\\\\\n", "&\\text{ for } k=0, \\cdots , n-1,\\\\\\\\\n", "r_E&=30,\\\\\n", "r_I&=20.\\\\\n", "\\end{align*}\n", "\n", "where $\\color{red}{r_E[k+1]}$ and $\\color{red}{r_I[k+1]}$ are unknown, $\\color{green}{r_E[k]}$ and $\\color{green}{r_I[k]}$ are known, $\\color{blue}{w_{EI}}=-1$, $\\color{blue}{w_{IE}}=1$ and $\\color{blue}{\\tau_E}=100ms$ and $\\color{blue}{\\tau_I}=120ms$ are known parameters and $\\color{blue}{\\Delta t}=0.01ms$ is a chosen time-step.\n", "__Code the difference equations to estimate $r_{E}$ and $r_{I}$__.\n", "\n", "Note that the equations have to estimated in the same for loop as they depend on each other." ] }, { "cell_type": "code", "execution_count": null, "id": "e1514105", "metadata": { "execution": {} }, "outputs": [], "source": [ "def Euler_Simple_Linear_System(t, dt):\n", " \"\"\"\n", " Args:\n", " time: time\n", " dt : time-step\n", " Returns:\n", " r_E : Excitation Firing Rate\n", " r_I : Inhibition Firing Rate\n", "\n", " \"\"\"\n", "\n", " # Set up parameters\n", " tau_E = 100\n", " tau_I = 120\n", " n = len(t)\n", " r_I = np.zeros(n)\n", " r_I = 20\n", " r_E = np.zeros(n)\n", " r_E = 30\n", "\n", " #######################################################################\n", " ## TODO for students: calculate the estimate solutions of r_E and r_I at t[i+1]\n", " ## Complete line of codes for dr_E and dr_I and remove\n", " raise NotImplementedError(\"Student exercise: calculate the estimate solutions of r_E and r_I at t[i+1]\")\n", " ########################################################################\n", "\n", " # Loop over time steps\n", " for k in range(n-1):\n", "\n", " # Estimate r_e\n", " dr_E = ...\n", " r_E[k+1] = r_E[k] + dt*dr_E\n", "\n", " # Estimate r_i\n", " dr_I = ...\n", " r_I[k+1] = r_I[k] + dt*dr_I\n", "\n", " return r_E, r_I\n", "\n", "# Set up dt, t\n", "dt = 0.1 # time-step\n", "t = np.arange(0, 1000, dt)\n", "\n", "# Run Euler method\n", "r_E, r_I = Euler_Simple_Linear_System(t, dt)\n", "\n", "# Visualize\n", "plot_rErI(t, r_E, r_I)" ] }, { "cell_type": "markdown", "id": "JnLfmtc5ZWXz", "metadata": { "colab_type": "text", "execution": {} }, "source": [ "[*Click for solution*](https://github.com/NeuromatchAcademy/precourse/tree/main/tutorials/W0D4_Calculus/solutions/W0D4_Tutorial3_Solution_77fad457.py)\n", "\n", "*Example output:*\n", "\n", " \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "ZGYMiHlAkHbV", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Euler_on_a_simple_system_Exercise\")" ] }, { "cell_type": "markdown", "id": "Fk-EvkAOImKR", "metadata": { "execution": {} }, "source": [ "### Think! 3.1: Simple Euler solution to the Wilson Cowan model\n", "\n", "1. Is the simulation biologically plausible?\n", "2. What is the effect of combined excitation and inhibition?\n" ] }, { "cell_type": "markdown", "id": "i3L1OGeUgZEe", "metadata": { "colab_type": "text", "execution": {} }, "source": [ "[*Click for solution*](https://github.com/NeuromatchAcademy/precourse/tree/main/tutorials/W0D4_Calculus/solutions/W0D4_Tutorial3_Solution_71a8704e.py)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "gyCVFYRHkIMH", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Simple_Euler_solution_to_the_Wilson_Cowan_model_Discussion\")" ] }, { "cell_type": "markdown", "id": "Cw-R4Yd4QCTH", "metadata": { "execution": {} }, "source": [ "## Section 3.2: Phase Plane Plot and Nullcline\n", "\n", "\n", "*Estimated timing to here from start of tutorial: 50 min*\n", "\n", "
\n", " Click here for text recap of relevant part of video \n", "\n", "When there are two differential equations describing the interaction of two processes like excitation $r_{E}$ and inhibition $r_{I}$ that are dependent on each other they can be plotted as a function of each other, which is known as a phase plane plot. The phase plane plot can give insight give insight into the state of the system but more about that later in Neuromatch Academy.\n", "\n", "In the animated figure below, the panel of the left shows the excitation firing rate $r_E$ and the inhibition firing rate $r_I$ as a function of time. The panel on the right hand side is the phase plane plot which shows the inhibition firing rate $r_I$ as a function of excitation firing rate $r_E$.\n", "\n", "An addition to the phase plane plot are the \"nullcline\". These lines plot when the rate of change $\\frac{d}{dt}$ of the variables is equal to $0$. We saw a variation of this for a single differential equation in the differential equation tutorial.\n", "\n", "As we have two differential equations we set $\\frac{dr_E}{dt}=0$ and $\\frac{dr_I}{dt}=0$ which gives the equations,\n", "\n", "\\begin{align}\n", "0&= w_{EI}r_I, \\\\\n", "0&= w_{IE}r_E,\\\\\n", "\\end{align}\n", "\n", "these are a unique example as they are a vertical and horizontal line. Where the lines cross is the stable point which the $r_E(t)$ excitatory population and $r_I(t)$ the inhibitory population orbit around." ] }, { "cell_type": "markdown", "id": "2ghDuf-YeG6z", "metadata": { "execution": {} }, "source": [ "" ] }, { "cell_type": "markdown", "id": "98f3c696", "metadata": { "execution": {} }, "source": [ "### Think! 6.1: Discuss the Plots\n", "\n", "1. Which representation is more intuitive (and useful), the time plot or the phase plane plot?\n", "2. Why do we only see one circle?\n", "3. What do the quadrants represent?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Execute the code to plot the solution to the system\n" ] }, { "cell_type": "code", "execution_count": null, "id": "036d3fb5", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown Execute the code to plot the solution to the system\n", "plot_rErI_Simple(t, r_E, r_I)" ] }, { "cell_type": "markdown", "id": "Sfa1foAYhB13", "metadata": { "colab_type": "text", "execution": {} }, "source": [ "[*Click for solution*](https://github.com/NeuromatchAcademy/precourse/tree/main/tutorials/W0D4_Calculus/solutions/W0D4_Tutorial3_Solution_7e9ca0da.py)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "nDKgymrbkJJ9", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Discuss_the_Plots_Discussion\")" ] }, { "cell_type": "markdown", "id": "5a278386", "metadata": { "execution": {} }, "source": [ "## Section 3.3: Adding internal connections\n", "\n", "*Estimated timing to here from start of tutorial: 57 min*\n", "\n", "Building up the equations in the previous section we re-introduce internal connections $w_{EE}$, $w_{II}$. The two coupled differential equations, each representing the dynamics of the excitatory or inhibitory population are now:\n", "\n", "\\begin{align}\n", "\\tau_E \\frac{dr_E}{dt} &=w_{EE}r_E +w_{EI}r_I, \\\\\n", "\\tau_I \\frac{dr_I}{dt} &=w_{IE}r_E +w_{II}r_I ,\n", "\\end{align}\n", "\n", "where $\\tau_E=100$ and $\\tau_I=120$, $w_{EE}=1$ and $w_{II}=-1$, and $w_{EI}=-5$ and $w_{IE}=0.6$,\n", "with the initial conditions\n", "\n", "\\begin{align}\n", "r_E(0)=30, \\\\\n", "r_I(0)=20.\n", "\\end{align}\n", "\n", "The solutions can be approximated using the Euler method such that the equations become:\n", "\n", "\\begin{align*}\n", "\\frac{\\color{red}{rE_{k+1}}-\\color{green}{rE_k}}{\\color{blue}{\\Delta t}}&=\\big(\\frac{\\color{blue}{w_{EE}}\\color{green}{rE_k}+\\color{blue}{w_{EI}}\\color{green}{rI_k}}{\\color{blue}{\\tau_E}}\\big),\\\\\n", "\\frac{\\color{red}{rI_{k+1}}-\\color{green}{rI_k}}{\\color{blue}{\\Delta t}}&=\\big(\\frac{\\color{blue}{w_{II}}\\color{green}{rI_k}+\\color{blue}{w_{IE}}\\color{green}{rE_k}}{\\color{blue}{\\tau_I}}\\big),\\\\\n", "\\end{align*}\n", "where $r_E[k]$ and $r_I[k]$ are the numerical estimates of the firing rate of the excitation population $r_E(t_k)$ and inhibition population $r_I(t_K)$ and $\\Delta t$ is the time-step.\n", "\n", "Re-arranging the equation such that all the known terms are on the right gives:\n", "\n", "\\begin{align*}\n", "\\color{red}{rE_{k+1}}&=\\color{green}{rE_k}+\\color{blue}{\\Delta t}\\big(\\frac{\\color{blue}{w_{EE}}\\color{green}{rE_k}+\\color{blue}{w_{EI}}\\color{green}{rI_k}}{\\color{blue}{\\tau_E}}\\big),\\\\\n", "\\color{red}{rI_{k+1}}&=\\color{green}{rI_k}+\\color{blue}{\\Delta t}\\big(\\frac{\\color{blue}{w_{II}}\\color{green}{rI_k}+\\color{blue}{w_{IE}}\\color{green}{rE_k}}{\\color{blue}{\\tau_I}}\\big),\\\\\n", "&\\text{ for } k=0, \\cdots n-1,\\\\\\\\\n", "rE_0&=30,\\\\\n", "rI_0&=20.\\\\\n", "\\end{align*}\n", "\n", "where $\\color{red}{rE_{k+1}}$ and $\\color{red}{rI_{k+1}}$ are unknown, $\\color{green}{rE_{k}}$ and $\\color{green}{rI_{k}}$ are known, $\\color{blue}{w_{EI}}=-1$, $\\color{blue}{w_{IE}}=1$, $\\color{blue}{w_{EE}}=1$, $\\color{blue}{w_{II}}=-1$ and $\\color{blue}{\\tau_E}=100$ and $\\color{blue}{\\tau_I}=120$ are known parameters and $\\color{blue}{\\Delta t}=0.1$ is a chosen time-step." ] }, { "cell_type": "markdown", "id": "byUkHQNsJDF-", "metadata": { "execution": {} }, "source": [ "### Think! 3.3: Oscillations\n", "\n", "\n", "The code below implements and visualizes the linear Wilson-Cowan model.\n", "\n", "1. What will happen to the oscillations if the time period is extended?\n", "2. How would you control or predict the oscillations?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " *Execute this cell to visualize the Linear Willson-Cowan*\n" ] }, { "cell_type": "code", "execution_count": null, "id": "e00d88fb", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown *Execute this cell to visualize the Linear Willson-Cowan*\n", "\n", "def Euler_Linear_System_Matrix(t, dt, w_EE=1):\n", " \"\"\"\n", " Args:\n", " time: time\n", " dt: time-step\n", " w_EE: Excitation to excitation weight\n", " Returns:\n", " r_E: Excitation Firing Rate\n", " r_I: Inhibition Firing Rate\n", " N_Er: Nullclines for drE/dt=0\n", " N_Ir: Nullclines for drI/dt=0\n", " \"\"\"\n", "\n", " tau_E = 100\n", " tau_I = 120\n", " n = len(t)\n", " r_I = 20*np.ones(n)\n", " r_E = 30*np.ones(n)\n", " w_EI = -5\n", " w_IE = 0.6\n", " w_II = -1\n", "\n", " for k in range(n-1):\n", "\n", " # Calculate the derivatives of the E and I populations\n", " drE = (w_EI*r_I[k] + w_EE*r_E[k]) / tau_E\n", " r_E[k+1] = r_E[k] + dt*drE\n", "\n", " drI = (w_II*r_I[k] + w_IE*r_E[k]) / tau_I\n", " r_I[k+1] = r_I[k] + dt*drI\n", "\n", "\n", " N_Er = -w_EE / w_EI*r_E\n", " N_Ir = -w_IE / w_II*r_E\n", "\n", " return r_E, r_I, N_Er, N_Ir\n", "\n", "\n", "dt = 0.1 # time-step\n", "t = np.arange(0, 1000, dt)\n", "r_E, r_I, _, _ = Euler_Linear_System_Matrix(t, dt)\n", "plot_rErI(t, r_E, r_I)" ] }, { "cell_type": "markdown", "id": "1ACUE5rriyyR", "metadata": { "colab_type": "text", "execution": {} }, "source": [ "[*Click for solution*](https://github.com/NeuromatchAcademy/precourse/tree/main/tutorials/W0D4_Calculus/solutions/W0D4_Tutorial3_Solution_9c8302d7.py)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "z4gwxcvJkKKN", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Oscillations_Discussion\")" ] }, { "cell_type": "markdown", "id": "261b8f71", "metadata": { "execution": {} }, "source": [ "## Section 3.4 Phase Plane and Nullclines Part 2\n", "\n", "*Estimated timing to here from start of tutorial: 62 min*\n", "\n", "Like before, we have two differential equations so we can plot the results on a phase plane. We can also calculate the Nullclines when we set $\\frac{dr_E}{dt}=0$ and $\\frac{dr_I}{dt}=0$ which gives,\n", "\\begin{align}\n", "0&= w_{EE}r_E+w_{EI}r_I, \\\\\n", "0&= w_{IE}r_E+w_{II}r_I,\n", "\\end{align}\n", "re-arranging as two lines\n", "\\begin{align}\n", "r_I&= -\\frac{w_{EE}}{w_{EI}}r_E, \\\\\n", "r_I&= -\\frac{w_{IE}}{w_{II}}r_E,\n", "\\end{align}\n", "which crosses at the stable point.\n", "\n", "The panel on the left shows excitation firing rate $r_E$ and inhibition firing rate $r_I$ as a function of time. On the right side the phase plane plot shows inhibition firing rate $r_I$ as a function of excitation firing rate $r_E$ with the Nullclines for $\\frac{dr_E}{dt}=0$ and $\\frac{dr_I}{dt}=0.$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " *Run this cell to visualize the phase plane*\n" ] }, { "cell_type": "code", "execution_count": null, "id": "3ca5f555", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown *Run this cell to visualize the phase plane*\n", "dt = 0.1 # time-step\n", "t = np.arange(0, 1000, dt)\n", "r_E, r_I, N_Er, N_Ir = Euler_Linear_System_Matrix(t, dt)\n", "plot_rErI_Matrix(t, r_E, r_I, N_Er, N_Ir)" ] }, { "cell_type": "markdown", "id": "4d220e96", "metadata": { "execution": {} }, "source": [ "### Interactive Demo 3.4: A small change changes everything\n", "\n", "We will illustrate that even changing one parameter in a system of differential equations can have a large impact on the solutions of the excitation firing rate $r_E$ and inhibition firing rate $r_I$.\n", "Interact with the widget below to change the size of $w_{EE}$.\n", "\n", "Take note of:\n", "1. How the solution changes for positive and negative of $w_{EE}$. Pay close attention to the axis.\n", "2. How would you maintain a stable oscillation." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Make sure you execute this cell to enable the widget!\n" ] }, { "cell_type": "code", "execution_count": null, "id": "42724aca", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown Make sure you execute this cell to enable the widget!\n", "my_layout.width = '450px'\n", "@widgets.interact(\n", " w_EE=widgets.FloatSlider(1, min=-1., max=2., step=.1,\n", " layout=my_layout)\n", ")\n", "\n", "def Pop_widget(w_EE):\n", " dt = 0.1 # time-step\n", " t = np.arange(0,1000,dt)\n", " r_E, r_I, N_Er, N_Ir = Euler_Linear_System_Matrix(t, dt, w_EE)\n", " plot_rErI_Matrix(t, r_E, r_I, N_Er, N_Ir)\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "sH4Bpv7lkLFe", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Small_change_changes_everything_Interactive_Demo\")" ] }, { "cell_type": "markdown", "id": "1268fd80", "metadata": { "execution": {} }, "source": [ "---\n", "# Summary\n", "\n", "*Estimated timing of tutorial: 70 minutes*\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Video 5: Summary\n" ] }, { "cell_type": "code", "execution_count": null, "id": "cEcFr2llcl-l", "metadata": { "cellView": "form", "execution": {}, "tags": [ "remove-input" ] }, "outputs": [], "source": [ "# @title Video 5: Summary\n", "from ipywidgets import widgets\n", "from IPython.display import YouTubeVideo\n", "from IPython.display import IFrame\n", "from IPython.display import display\n", "\n", "\n", "class PlayVideo(IFrame):\n", " def __init__(self, id, source, page=1, width=400, height=300, **kwargs):\n", " self.id = id\n", " if source == 'Bilibili':\n", " src = f'https://player.bilibili.com/player.html?bvid={id}&page={page}'\n", " elif source == 'Osf':\n", " src = f'https://mfr.ca-1.osf.io/render?url=https://osf.io/download/{id}/?direct%26mode=render'\n", " super(PlayVideo, self).__init__(src, width, height, **kwargs)\n", "\n", "\n", "def display_videos(video_ids, W=400, H=300, fs=1):\n", " tab_contents = []\n", " for i, video_id in enumerate(video_ids):\n", " out = widgets.Output()\n", " with out:\n", " if video_ids[i] == 'Youtube':\n", " video = YouTubeVideo(id=video_ids[i], width=W,\n", " height=H, fs=fs, rel=0)\n", " print(f'Video available at https://youtube.com/watch?v={video.id}')\n", " else:\n", " video = PlayVideo(id=video_ids[i], source=video_ids[i], width=W,\n", " height=H, fs=fs, autoplay=False)\n", " if video_ids[i] == 'Bilibili':\n", " print(f'Video available at https://www.bilibili.com/video/{video.id}')\n", " elif video_ids[i] == 'Osf':\n", " print(f'Video available at https://osf.io/{video.id}')\n", " display(video)\n", " tab_contents.append(out)\n", " return tab_contents\n", "\n", "\n", "video_ids = [('Youtube', '5UmGgboSc40'), ('Bilibili', 'BV1wM4y1g78M')]\n", "tab_contents = display_videos(video_ids, W=730, H=410)\n", "tabs = widgets.Tab()\n", "tabs.children = tab_contents\n", "for i in range(len(tab_contents)):\n", " tabs.set_title(i, video_ids[i])\n", "display(tabs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "uzv5CG0AkLwR", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Summary_Video\")" ] }, { "cell_type": "markdown", "id": "-JZJWqTnitQx", "metadata": { "execution": {} }, "source": [ "Using the formula for the slope of a line, the solution of the differential equation can be estimated with reasonable accuracy. This will be much more relevant when dealing with more complicated (non-linear) differential equations where there is no known exact solution." ] }, { "cell_type": "markdown", "id": "a307a8fa", "metadata": { "execution": {} }, "source": [ "---\n", "## Links to Neuromatch Computational Neuroscience Days\n", "\n", "Differential equations turn up on a number of different Neuromatch days:\n", "* The LIF model is discussed in more detail in [Model Types](https://compneuro.neuromatch.io/tutorials/W1D1_ModelTypes/chapter_title.html) and [Biological Neuron Models](https://compneuro.neuromatch.io/tutorials/W2D3_BiologicalNeuronModels/chapter_title.html).\n", "* Drift Diffusion model, which is a differential equation for decision making, is discussed in [Linear Systems](https://compneuro.neuromatch.io/tutorials/W2D2_LinearSystems/chapter_title.html).\n", "* Phase-plane plots are discussed in [Linear Systems](https://compneuro.neuromatch.io/tutorials/W2D2_LinearSystems/chapter_title.html) and [Dynamic Networks](https://compneuro.neuromatch.io/tutorials/W2D4_DynamicNetworks/chapter_title.html).\n", "* The Wilson-Cowan model is discussed in [Dynamic Networks](https://compneuro.neuromatch.io/tutorials/W2D4_DynamicNetworks/chapter_title.html).\n", "\n", "## References\n", "1. Lotka, A. L, (1920) Analytical note on certain rhythmic relations inorganic systems.Proceedings of the National Academy of Sciences, 6(7):410–415,1920. doi: [10.1073/pnas.6.7.410](https://doi.org/10.1073/pnas.6.7.410)\n", "2. Brunel N, van Rossum MC. Lapicque's 1907 paper: from frogs to integrate-and-fire. Biol Cybern. 2007 Dec;97(5-6):337-9. doi: [10.1007/s00422-007-0190-0](https://doi.org/10.1007/s00422-007-0190-0). Epub 2007 Oct 30. PMID: 17968583.\n", "\n", "\n", "## Bibliography\n", "1. Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience: computational and mathematical modeling of neural systems. Computational Neuroscience Series.\n", "2. Strogatz, S. (2014). Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering (studies in nonlinearity), Westview Press; 2nd edition\n", "\n", "### Supplemental Popular Reading List\n", "1. Lindsay, G. (2021). Models of the Mind: How Physics, Engineering and Mathematics Have Shaped Our Understanding of the Brain. Bloomsbury Publishing.\n", "2. Strogatz, S. (2004). Sync: The emerging science of spontaneous order. Penguin UK.\n", "\n", "### Popular Podcast\n", "1. Strogatz, S. (Host). (2020), Joy of X https://www.quantamagazine.org/tag/the-joy-of-x/ Quanta Magazine" ] }, { "cell_type": "markdown", "id": "_bnKhPLwobY9", "metadata": { "execution": {} }, "source": [ "---\n", "# Bonus" ] }, { "cell_type": "markdown", "id": "3c9b3f6e", "metadata": { "execution": {} }, "source": [ "---\n", "## Bonus Section 1: The 4th Order Runge-Kutta method\n", "\n", "Another popular numerical method to estimate the solution of differential equations of the general form:\n", "\n", "\\begin{align}\n", "\\frac{d}{dt}y=f(t,y)\n", "\\end{align}\n", "\n", "is the 4th Order Runge Kutta:\n", "\\begin{align}\n", "k_1 &= f(t_k,y_k)\\\\\n", "k_2 &= f(t_k+\\frac{\\Delta t}{2},y_k+\\frac{\\Delta t}{2}k_1)\\\\\n", "k_3 &= f(t_k+\\frac{\\Delta t}{2},y_k+\\frac{\\Delta t}{2}k_2)\\\\\n", "k_4 &= f(t_k+\\Delta t,y_k+\\Delta tk_3)\\\\\n", "y_{k+1} &= y_{k}+\\frac{\\Delta t}{6}(k_1+2k_2+2k_3+k_4)\\\\\n", "\\end{align}\n", "\n", "for $k=0,1,\\cdots,n-1$,\n", "which is more accurate than the Euler method.\n", "\n", "Given the population differential equation\n", "\\begin{align*}\n", "\\frac{d}{dt}\\,p(t) &= 0.3 p(t),\\\\\n", "p(t_0=1)&=e^{0.3}\\quad \\text{Initial Condition},\n", "\\end{align*}\n", "\n", "the 4th Runge Kutta difference equation is\n", "\n", "\\begin{align*}\n", "k_1&=0.3\\color{green}{p[k]},\\\\\n", "k_2&=0.3(\\color{green}{p[k]}+\\frac{\\Delta t}{2}k_1),\\\\\n", "k_3&=0.3(\\color{green}{p[k]}+\\frac{\\Delta t}{2}k_2),\\\\\n", "k_4&=0.3(\\color{green}{p[k]}+k_3),\\\\\n", "\\color{red}{p_{k[k]}}&=\\color{green}{p[k]}+\\frac{\\color{blue}{\\Delta t}}{6}( \\color{green}{k_1+2k_2+2k_3+k_4})\\\\\n", "\\end{align*}\n", "\n", "for $k=0,1,\\cdots 4$ to estimate the population for $\\Delta t=0.5$.\n", "\n", "The code is implemented below. Note how much more accurate the 4th Order Runge Kutta (yellow) is compared to the Euler Method (blue). The 4th order Runge Kutta is of order 4 which means that if you half the time-step $\\Delta t$ the error decreases by a factor of $\\Delta t^4$." ] }, { "cell_type": "code", "execution_count": null, "id": "7b936428", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown *Execute this cell to show the difference between the Runge Kutta Method and the Euler Method*\n", "\n", "\n", "def RK4(dt=0.5):\n", " t=np.arange(1, 5+dt/2, dt)\n", " t_fine=np.arange(1, 5+0.1/2, 0.1)\n", " n = len(t)\n", " p = np.ones(n)\n", " pRK4 = np.ones(n)\n", " p = np.exp(0.3*t)\n", " pRK4 = np.exp(0.3*t)# Initial Condition\n", "\n", " with plt.xkcd():\n", "\n", " fig = plt.figure(figsize=(6, 4))\n", " plt.plot(t, np.exp(0.3*t), 'k', label='Exact Solution')\n", "\n", " for i in range(n-1):\n", " dp = dt*0.3*p[i]\n", " p[i+1] = p[i] + dp # Euler\n", " k1 = 0.3*(pRK4[i])\n", " k2 = 0.3*(pRK4[i] + dt/2*k1)\n", " k3 = 0.3*(pRK4[i] + dt/2*k2)\n", " k4 = 0.3*(pRK4[i] + dt*k3)\n", " pRK4[i+1] = pRK4[i] + dt/6 *(k1 + 2*k2 + 2*k3 + k4)\n", "\n", " plt.plot(t_fine, np.exp(0.3*t_fine), label='Exact')\n", " plt.plot(t, p,':ro', label='Euler Estimate')\n", " plt.plot(t, pRK4,':co', label='4th Order Runge Kutta Estimate')\n", "\n", " plt.ylabel('Population (millions)')\n", " plt.legend()\n", " plt.xlabel('Time (years)')\n", " plt.show()\n", "\n", "# @markdown Make sure you execute this cell to enable the widget!\n", "my_layout.width = '450px'\n", "@widgets.interact(\n", " dt=widgets.FloatSlider(0.5, min=.1, max=4., step=.1,\n", " layout=my_layout)\n", ")\n", "def Pop_widget(dt):\n", " RK4(dt)" ] }, { "cell_type": "markdown", "id": "W7ziQulJL6Kl", "metadata": { "execution": {} }, "source": [ "**Bonus Reference 1: A full course on numerical methods in Python**\n", "\n", "For a full course on Numerical Methods for differential Equations you can look [here](https://github.com/john-s-butler-dit/Numerical-Analysis-Python)." ] }, { "cell_type": "markdown", "id": "jXpJytN7dgYn", "metadata": { "execution": {} }, "source": [ "---\n", "## Bonus Section 2: Neural oscillations are a start toward understanding brain activity rather than the end\n", "\n", "The differential equations we have discussed above are all to simulate neuronal processes. Another way differential equations can be used is to motivate experimental findings.\n", "\n", "Many experimental and neurophysiological studies have investigated whether the brain oscillates and/or entrees to a stimulus.\n", "An issue with these studies is that there is no consistent definition of what constitutes an oscillation. Right now, it is a bit of I know one when I see one problem.\n", "\n", "In an essay from May 2021 in PLoS Biology, Keith Dowling (a past Neuromatch TA) and M. Florencia Assaneo discussed a mathematical way of thinking about what should be expected experimentally when studying oscillations. The essay proposes that instead of thinking about the brain, we should look at this question from the mathematical side to motivate what can be defined as an oscillation.\n", "\n", "To do this, they used Stuart–Landau equations, which is a system of differential equations\n", "\\begin{align}\n", "\\frac{dx}{dt} &= \\lambda x-\\omega y -\\gamma (x^2+y^2)x+s\\\\\n", "\\frac{dy}{dt} &= \\lambda y+\\omega x -\\gamma (x^2+y^2)y\n", "\\end{align}\n", "\n", "where $s$ is input to the system, and $\\lambda$, $\\omega$ and $\\gamma$ are parameters.\n", "\n", "The Stuart–Landau equations are a well-described system of non-linear differential equations that generate oscillations. For their purpose, Dowling and Assaneo used the equations to motivate what experimenters should expect when conducting experiments looking for oscillations.\n", "In their paper, using the Stuart–Landau equations, they outline\n", "* \"What is an oscillator?\"\n", "* \"What an oscillator is not.\"\n", "* \"Not all that oscillates is an oscillator.\"\n", "* \"Not all oscillators are alike.\"\n", "\n", "The Euler form of the Stuart–Landau system of equations is:\n", "\n", "\\begin{align*}\n", "\\color{red}{x_{k+1}}&=\\color{green}{x_k}+\\color{blue}{\\Delta t}\\big(\\lambda \\color{green}{x_k}-\\omega \\color{green}{y_k} -\\gamma (\\color{green}{x_k}^2+\\color{green}{y_k}^2)\\color{green}{x_k}+s\\big),\\\\\n", "\\color{red}{y_{k+1}}&=\\color{green}{y_k}+\\color{blue}{\\Delta t}\\big(\\lambda \\color{green}{y_k}+\\omega \\color{green}{x_k} -\\gamma (\\color{green}{x_k}^2+\\color{green}{y_k}^2)\\color{green}{y_k} \\big),\\\\\n", "&\\text{ for } k=0, \\cdots n-1,\\\\\n", "x_0&=1,\\\\\n", "y_0&=1,\\\\\n", "\\end{align*}\n", "\n", "with $\\Delta t=0.1/1000$ ms." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Helper functions\n" ] }, { "cell_type": "code", "execution_count": null, "id": "5b9e30cc", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Helper functions\n", "def plot_Stuart_Landa(t, x, y, s):\n", " \"\"\"\n", " Args:\n", " t : time\n", " x : x\n", " y : y\n", " s : input\n", " Returns:\n", " figure with two panels\n", " top panel: Input as a function of time\n", " bottom panel: x\n", " \"\"\"\n", "\n", " with plt.xkcd():\n", " fig = plt.figure(figsize=(14, 4))\n", " gs = gridspec.GridSpec(2, 2, height_ratios=[1, 4], width_ratios=[4, 1])\n", "\n", " # PLOT OF INPUT\n", " plt.subplot(gs)\n", " plt.ylabel(r'$s$')\n", " plt.plot(t, s, 'g')\n", " #plt.ylim((2,4))\n", "\n", " # PLOT OF ACTIVITY\n", " plt.subplot(gs)\n", " plt.plot(t ,x)\n", " plt.ylabel(r'x')\n", " plt.xlabel(r't')\n", " plt.subplot(gs)\n", " plt.plot(x,y)\n", " plt.plot(x, y,'go')\n", " plt.xlabel(r'x')\n", " plt.ylabel(r'y')\n", " plt.show()\n", "\n", "\n", "def Euler_Stuart_Landau(s,time,dt,lamba=0.1,gamma=1,k=25):\n", " \"\"\"\n", " Args:\n", " I: Input\n", " time: time\n", " dt: time-step\n", " \"\"\"\n", "\n", " n = len(time)\n", " omega = 4 * 2*np.pi\n", " x = np.zeros(n)\n", " y = np.zeros(n)\n", " x = 1\n", " y = 1\n", "\n", " for i in range(n-1):\n", " dx = lamba*x[i] - omega*y[i] - gamma*(x[i]*x[i] + y[i]*y[i])*x[i] + k*s[i]\n", " x[i+1] = x[i] + dt*dx\n", " dy = lamba*y[i] + omega*x[i] - gamma*(x[i]*x[i] + y[i]*y[i])*y[i]\n", " y[i+1] = y[i] + dt*dy\n", "\n", " return x, y" ] }, { "cell_type": "markdown", "id": "H5inj2EKJyqN", "metadata": { "execution": {} }, "source": [ "### Bonus 2.1: What is an Oscillator?\n", "\n", "Doelling & Assaneo (2021), using the Stuart–Landau system, show different possible states of an oscillator by manipulating the $\\lambda$ term in the equation.\n", "\n", "From the paper:\n", "\n", "> This qualitative change in behavior takes place at $\\lambda = 0$. For $\\lambda < 0$, the system decays to a stable equilibrium, while for $\\lambda > 0$, it keeps oscillating.\n", "\n", "This illustrates that oscillations do not have to maintain all the time, so experimentally we should not expect perfectly maintained oscillations. We see this all the time in $\\alpha$ band oscillations in EEG the oscillations come and go." ] }, { "cell_type": "markdown", "id": "5MyIOK9w_qjd", "metadata": { "execution": {} }, "source": [ "#### Interactive Demo Bonus 2.1: Oscillator\n", "\n", "The plot below shows the estimated solution of the Stuart–Landau system with a base frequency $\\omega$ set to $4\\times 2\\pi$, $\\gamma=1$ and $k=25$ over 3 seconds. The input to the system $s(t)$ is plotted in the top panel, $x$ as a function of time in the the bottom panel and on the right the phase plane plot of $x$ and $y$. You can manipulate $\\lambda$ to see how the oscillations change." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Make sure you execute this cell to enable the widget!\n" ] }, { "cell_type": "code", "execution_count": null, "id": "LAnBGRU5crfG", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown Make sure you execute this cell to enable the widget!\n", "dt=0.1/1000\n", "t=np.arange(0, 3, dt)\n", "\n", "my_layout.width = '450px'\n", "@widgets.interact(\n", " lamda=widgets.FloatSlider(1, min=-1., max=5., step=0.5,\n", " layout=my_layout)\n", ")\n", "\n", "def Pop_widget(lamda):\n", " s=np.zeros(len(t))\n", " x,y=Euler_Stuart_Landau(s,t,dt,lamda)\n", " plot_Stuart_Landa(t, x, y, s)\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "PNtk7NRomDuQ", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Oscillator_Bonus_Interactive_Demo\")" ] }, { "cell_type": "markdown", "id": "q2OMPLOoKGtr", "metadata": { "execution": {} }, "source": [ "### Bonus 2.2 : Not all oscillators are alike" ] }, { "cell_type": "markdown", "id": "NLOVdwN5_ypj", "metadata": { "execution": {} }, "source": [ "#### Interactive Demo Bonus 2: Stuart-Landau System\n", "\n", "The plot below shows the estimated solution of the Stuart–Landau system with a base frequency of 4Hz by setting $\\omega$ to $4\\times 2\\pi$, $\\lambda=1$, $\\gamma=1$ and $k=50$ over 3 seconds the input to the system $s(t)=\\sin(freq 2\\pi t)$ in the top panel, $x$ as a function of time in the bottom panel and on the right the phase plane plot of $x$ and $y$.\n", "\n", "You can manipulate the frequency $freq$ of the input to see how the oscillations change, and for frequencies $freq$ further and further from the base frequency of 4Hz, the oscillations break down.\n", "\n", "This shows that if you have an oscillating input into an oscillator, it does not have to respond by oscillating about could even break down. Hence the frequency of the input oscillation is important to the system.\n", "So if you flash something at 50Hz, for example, the visual system might not follow the signal, but that does not mean the visual system is not an oscillator. It might just be the wrong frequency." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Make sure you execute this cell to enable the widget!\n" ] }, { "cell_type": "code", "execution_count": null, "id": "XPuVG5f4clxt", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @markdown Make sure you execute this cell to enable the widget!\n", "dt=0.1/1000\n", "t=np.arange(0, 3, dt)\n", "\n", "my_layout.width = '450px'\n", "@widgets.interact(\n", " freq=widgets.FloatSlider(4, min=0.5, max=10., step=0.5,\n", " layout=my_layout)\n", ")\n", "\n", "def Pop_widget(freq):\n", " s = np.sin(freq * 2*np.pi * t)\n", "\n", " x, y = Euler_Stuart_Landau(s, t, dt, lamba=1, gamma=.1, k=50)\n", " plot_Stuart_Landa(t, x, y, s)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Submit your feedback\n" ] }, { "cell_type": "code", "execution_count": null, "id": "EC5oefXvmOtd", "metadata": { "cellView": "form", "execution": {}, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "# @title Submit your feedback\n", "content_review(f\"{feedback_prefix}_Stuart_Landau_System_Bonus_Interactive_Demo\")" ] }, { "cell_type": "markdown", "id": "1cWYP3SNL1Mh", "metadata": { "execution": {} }, "source": [ "**Bonus Reference 2**:\n", "\n", "Doelling, K. B., & Assaneo, M. F. (2021). Neural oscillations are a start toward understanding brain activity rather than the end. PLoS biology, 19(5), e3001234. doi: [10.1371/journal.pbio.3001234](https://doi.org/10.1371/journal.pbio.3001234)" ] } ], "metadata": { "colab": { "collapsed_sections": [ "1cWYP3SNL1Mh" ], "include_colab_link": true, "name": "W0D4_Tutorial3", "provenance": [], "toc_visible": true }, "kernel": { "display_name": "Python 3", "language": "python", "name": "python3" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.17" } }, "nbformat": 4, "nbformat_minor": 5 }