{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial 5: Point-Neuron Network Models (with PointNet)\n",
    "\n",
    "In this chapter we will create a heterogeneous network of point-model neurons and use the PointNet simulator which will run the network using the NEST simulator. As with the previous BioNet examples will create both a internal recurrently-connected network of different node types, and an external network of \"virtual\" neurons that will drive the firing of the internal neurons. And we'll show how to drive network activity by using a current clamp.\n",
    "\n",
    "PointNet, like BioNet and the other simulators, use the SONATA data format for representing networks, setting up simulations and saving results. Thus the tools used to build and display biophysically detailed networks in the previous chapters will work just the same. \n",
    "\n",
    "Requirements:\n",
    "* bmtk\n",
    "* NEST 2.11+"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Building the network\n",
    "\n",
    "There are two ways of generating a network of point-neurons. Either we can take the existing biophysical network created in the previous chapters and make some minor adjustments to the neuron models being used. Or we can build a new network from scratch using the BMTK Builder.\n",
    "\n",
    "### Converting networks\n",
    "We want to take the BioNet V1 network and change parameters so that the individual neurons are using point models. Luckily there parameters are stored in the node and edge \"types\" csv files, thus we can easily change them with a simple text editor (emacs, vi, sublime-text, etc). Here is an example of the old *V1_node_types.csv*: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>node_type_id</th>\n",
       "      <th>ei</th>\n",
       "      <th>morphology_file</th>\n",
       "      <th>model_processing</th>\n",
       "      <th>pop_name</th>\n",
       "      <th>location</th>\n",
       "      <th>model_template</th>\n",
       "      <th>model_type</th>\n",
       "      <th>dynamics_params</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100</td>\n",
       "      <td>e</td>\n",
       "      <td>Scnn1a.swc</td>\n",
       "      <td>aibs_perisomatic</td>\n",
       "      <td>Scnn1a</td>\n",
       "      <td>L4</td>\n",
       "      <td>ctdb:Biophys1.hoc</td>\n",
       "      <td>biophysical</td>\n",
       "      <td>472363762_fit.json</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>101</td>\n",
       "      <td>i</td>\n",
       "      <td>Pvalb.swc</td>\n",
       "      <td>aibs_perisomatic</td>\n",
       "      <td>PV</td>\n",
       "      <td>L4</td>\n",
       "      <td>ctdb:Biophys1.hoc</td>\n",
       "      <td>biophysical</td>\n",
       "      <td>472912177_fit.json</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>102</td>\n",
       "      <td>e</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LIF_exc</td>\n",
       "      <td>VisL4</td>\n",
       "      <td>nrn:IntFire1</td>\n",
       "      <td>point_process</td>\n",
       "      <td>IntFire1_exc_1.json</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>103</td>\n",
       "      <td>i</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LIF_inh</td>\n",
       "      <td>VisL4</td>\n",
       "      <td>nrn:IntFire1</td>\n",
       "      <td>point_process</td>\n",
       "      <td>IntFire1_inh_1.json</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   node_type_id ei morphology_file  model_processing pop_name location  \\\n",
       "0           100  e      Scnn1a.swc  aibs_perisomatic   Scnn1a       L4   \n",
       "1           101  i       Pvalb.swc  aibs_perisomatic       PV       L4   \n",
       "2           102  e             NaN               NaN  LIF_exc    VisL4   \n",
       "3           103  i             NaN               NaN  LIF_inh    VisL4   \n",
       "\n",
       "      model_template     model_type      dynamics_params  \n",
       "0  ctdb:Biophys1.hoc    biophysical   472363762_fit.json  \n",
       "1  ctdb:Biophys1.hoc    biophysical   472912177_fit.json  \n",
       "2       nrn:IntFire1  point_process  IntFire1_exc_1.json  \n",
       "3       nrn:IntFire1  point_process  IntFire1_inh_1.json  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "pd.read_csv('sources/chapter05/converted_network/V1_node_types_bionet.csv', sep=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and here is the *V1_node_types.csv* used for PointNet:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>node_type_id</th>\n",
       "      <th>ei</th>\n",
       "      <th>morphology_file</th>\n",
       "      <th>model_processing</th>\n",
       "      <th>pop_name</th>\n",
       "      <th>location</th>\n",
       "      <th>model_template</th>\n",
       "      <th>model_type</th>\n",
       "      <th>dynamics_params</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100</td>\n",
       "      <td>e</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Scnn1a</td>\n",
       "      <td>L4</td>\n",
       "      <td>nest:iaf_psc_alpha</td>\n",
       "      <td>point_process</td>\n",
       "      <td>472363762_point.json</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>101</td>\n",
       "      <td>i</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PV</td>\n",
       "      <td>L4</td>\n",
       "      <td>nest:iaf_psc_alpha</td>\n",
       "      <td>point_process</td>\n",
       "      <td>472912177_point.json</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>102</td>\n",
       "      <td>e</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LIF_exc</td>\n",
       "      <td>VisL4</td>\n",
       "      <td>nest:iaf_psc_alpha</td>\n",
       "      <td>point_process</td>\n",
       "      <td>IntFire1_exc_point.json</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>103</td>\n",
       "      <td>i</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LIF_inh</td>\n",
       "      <td>VisL4</td>\n",
       "      <td>nest:iaf_psc_alpha</td>\n",
       "      <td>point_process</td>\n",
       "      <td>IntFire1_inh_point.json</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   node_type_id ei  morphology_file  model_processing pop_name location  \\\n",
       "0           100  e              NaN               NaN   Scnn1a       L4   \n",
       "1           101  i              NaN               NaN       PV       L4   \n",
       "2           102  e              NaN               NaN  LIF_exc    VisL4   \n",
       "3           103  i              NaN               NaN  LIF_inh    VisL4   \n",
       "\n",
       "       model_template     model_type          dynamics_params  \n",
       "0  nest:iaf_psc_alpha  point_process     472363762_point.json  \n",
       "1  nest:iaf_psc_alpha  point_process     472912177_point.json  \n",
       "2  nest:iaf_psc_alpha  point_process  IntFire1_exc_point.json  \n",
       "3  nest:iaf_psc_alpha  point_process  IntFire1_inh_point.json  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv('sources/chapter05/converted_network/V1_node_types.csv', sep=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changes:\n",
    "* **model_type** - PointNet will not support the \"biophysical\" model_type and only support \"point_process\" neuron models.\n",
    "* **model_template** - nrn:IntFire1 and ctdb:Biophys1.hoc are special directives for running NEURON based models. Instead we replaced them with the \"nest:\\<nest-model\\>\" directive (note we can replace iaf_psc_alpha with any valid NEST model).\n",
    "* **dynamics_params** - We have new json parameters files for the new NEST based models.\n",
    "* **model_processing** - \"aibs_perisomatic\" is a special command for adjusting the morphology of biophysical models, and since our NEST-based models do not have a morphology we set it to none which tells the simulator to use the models as-is (note: you can implement custom model_processing functions for PointNet that will be explained later)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We must also adjust the *edges_types.csv* files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>edge_type_id</th>\n",
       "      <th>target_query</th>\n",
       "      <th>source_query</th>\n",
       "      <th>syn_weight</th>\n",
       "      <th>dynamics_params</th>\n",
       "      <th>distance_range</th>\n",
       "      <th>delay</th>\n",
       "      <th>target_sections</th>\n",
       "      <th>weight_function</th>\n",
       "      <th>model_template</th>\n",
       "      <th>weight_sigma</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100</td>\n",
       "      <td>pop_name=='Scnn1a'</td>\n",
       "      <td>ei=='e'</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>ExcToExc.json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>gaussianLL</td>\n",
       "      <td>static_synapse</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>101</td>\n",
       "      <td>pop_name=='LIF_exc'</td>\n",
       "      <td>ei=='e'</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>instanteneousExc.json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>gaussianLL</td>\n",
       "      <td>static_synapse</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>102</td>\n",
       "      <td>model_type=='biophysical'&amp;ei=='i'</td>\n",
       "      <td>ei=='i'</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>InhToInh.json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>wmax</td>\n",
       "      <td>static_synapse</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>103</td>\n",
       "      <td>model_type=='point_process'&amp;ei=='i'</td>\n",
       "      <td>ei=='i'</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>instanteneousInh.json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>wmax</td>\n",
       "      <td>static_synapse</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>104</td>\n",
       "      <td>model_type=='biophysical'&amp;ei=='e'</td>\n",
       "      <td>ei=='i'</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>InhToExc.json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>wmax</td>\n",
       "      <td>static_synapse</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>105</td>\n",
       "      <td>model_type=='point_process'&amp;ei=='e'</td>\n",
       "      <td>ei=='i'</td>\n",
       "      <td>30.00000</td>\n",
       "      <td>instanteneousInh.json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>wmax</td>\n",
       "      <td>static_synapse</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>106</td>\n",
       "      <td>pop_name=='PV'</td>\n",
       "      <td>ei=='e'</td>\n",
       "      <td>0.00035</td>\n",
       "      <td>ExcToInh.json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>wmax</td>\n",
       "      <td>static_synapse</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>107</td>\n",
       "      <td>pop_name=='LIF_inh'</td>\n",
       "      <td>ei=='e'</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>instanteneousExc.json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>wmax</td>\n",
       "      <td>static_synapse</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   edge_type_id                         target_query source_query  syn_weight  \\\n",
       "0           100                   pop_name=='Scnn1a'      ei=='e'    50.00000   \n",
       "1           101                  pop_name=='LIF_exc'      ei=='e'    50.00000   \n",
       "2           102    model_type=='biophysical'&ei=='i'      ei=='i'    50.00000   \n",
       "3           103  model_type=='point_process'&ei=='i'      ei=='i'    50.00000   \n",
       "4           104    model_type=='biophysical'&ei=='e'      ei=='i'    50.00000   \n",
       "5           105  model_type=='point_process'&ei=='e'      ei=='i'    30.00000   \n",
       "6           106                       pop_name=='PV'      ei=='e'     0.00035   \n",
       "7           107                  pop_name=='LIF_inh'      ei=='e'    50.00000   \n",
       "\n",
       "         dynamics_params  distance_range  delay  target_sections  \\\n",
       "0          ExcToExc.json             NaN    2.0              NaN   \n",
       "1  instanteneousExc.json             NaN    2.0              NaN   \n",
       "2          InhToInh.json             NaN    2.0              NaN   \n",
       "3  instanteneousInh.json             NaN    2.0              NaN   \n",
       "4          InhToExc.json             NaN    2.0              NaN   \n",
       "5  instanteneousInh.json             NaN    2.0              NaN   \n",
       "6          ExcToInh.json             NaN    2.0              NaN   \n",
       "7  instanteneousExc.json             NaN    2.0              NaN   \n",
       "\n",
       "  weight_function  model_template  weight_sigma  \n",
       "0      gaussianLL  static_synapse          50.0  \n",
       "1      gaussianLL  static_synapse          50.0  \n",
       "2            wmax  static_synapse           NaN  \n",
       "3            wmax  static_synapse           NaN  \n",
       "4            wmax  static_synapse           NaN  \n",
       "5            wmax  static_synapse           NaN  \n",
       "6            wmax  static_synapse           NaN  \n",
       "7            wmax  static_synapse           NaN  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv('sources/chapter05/converted_network/V1_V1_edge_types.csv', sep=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* **model_template** has been changed to use a NEST based model type (static_synapse)\n",
    "* Use different **dynamics_parameter** files \n",
    "* It's important to readjust **syn_weight** as values appropiate for NEURON based models are oftern wrong for NEST based models. \n",
    "\n",
    "Notice we don't have to change any of the hdf5 files. The network topology remains the same making it a powerful tool for comparing networks of different levels of resolution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building a model from scratch.\n",
    "\n",
    "We can use the BMTK Network Builder to create new network files just for point-based modeling\n",
    "\n",
    "#### V1 Network\n",
    "\n",
    "First lets build a \"V1\" network of 300 cells, split into 4 different populations "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bmtk.builder.networks import NetworkBuilder\n",
    "from bmtk.builder.auxi.node_params import positions_columinar\n",
    "\n",
    "net = NetworkBuilder(\"V1\")\n",
    "net.add_nodes(N=80,  # Create a population of 80 neurons\n",
    "              positions=positions_columinar(N=80, center=[0, 50.0, 0], max_radius=30.0, height=100.0),\n",
    "              pop_name='Scnn1a', location='VisL4', ei='e',  # optional parameters\n",
    "              model_type='point_process',  # Tells the simulator to use point-based neurons\n",
    "              model_template='nest:iaf_psc_alpha',  # tells the simulator to use NEST iaf_psc_alpha models\n",
    "              dynamics_params='472363762_point.json'  # File containing iaf_psc_alpha mdoel parameters\n",
    "             )\n",
    "\n",
    "net.add_nodes(N=20, pop_name='PV', location='VisL4', ei='i',\n",
    "              positions=positions_columinar(N=20, center=[0, 50.0, 0], max_radius=30.0, height=100.0),\n",
    "              model_type='point_process',\n",
    "              model_template='nest:iaf_psc_alpha',\n",
    "              dynamics_params='472912177_point.json')\n",
    "\n",
    "net.add_nodes(N=200, pop_name='LIF_exc', location='L4', ei='e',\n",
    "              positions=positions_columinar(N=200, center=[0, 50.0, 0], min_radius=30.0, max_radius=60.0, height=100.0),\n",
    "              model_type='point_process',\n",
    "              model_template='nest:iaf_psc_alpha',\n",
    "              dynamics_params='IntFire1_exc_point.json')\n",
    "\n",
    "net.add_nodes(N=100, pop_name='LIF_inh', location='L4', ei='i',\n",
    "              positions=positions_columinar(N=100, center=[0, 50.0, 0], min_radius=30.0, max_radius=60.0, height=100.0),\n",
    "              model_type='point_process',\n",
    "              model_template='nest:iaf_psc_alpha',\n",
    "              dynamics_params='IntFire1_inh_point.json')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now go ahead and created synaptic connections then build and save our network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bmtk.builder.connection_map.ConnectionMap at 0x7ffbbc337748>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bmtk.builder.auxi.edge_connectors import distance_connector\n",
    "\n",
    "## E-to-E connections\n",
    "net.add_edges(source={'ei': 'e'}, target={'pop_name': 'Scnn1a'},\n",
    "              connection_rule=distance_connector,\n",
    "              connection_params={'d_weight_min': 0.0, 'd_weight_max': 0.34, 'd_max': 300.0, 'nsyn_min': 3, 'nsyn_max': 7},\n",
    "              syn_weight=3.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='ExcToExc.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "net.add_edges(source={'ei': 'e'}, target={'pop_name': 'LIF_exc'},\n",
    "              connection_rule=distance_connector,\n",
    "              connection_params={'d_weight_min': 0.0, 'd_weight_max': 0.34, 'd_max': 300.0, 'nsyn_min': 3, 'nsyn_max': 7},\n",
    "              syn_weight=3.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='instanteneousExc.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "\n",
    "### Generating I-to-I connections\n",
    "net.add_edges(source={'ei': 'i'}, target={'pop_name': 'PV'},\n",
    "              connection_rule=distance_connector,\n",
    "              connection_params={'d_weight_min': 0.0, 'd_weight_max': 1.0, 'd_max': 160.0, 'nsyn_min': 3, 'nsyn_max': 7},\n",
    "              syn_weight=-3.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='InhToInh.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "net.add_edges(source={'ei': 'i'}, target={'ei': 'i', 'pop_name': 'LIF_inh'},\n",
    "              connection_rule=distance_connector,\n",
    "              connection_params={'d_weight_min': 0.0, 'd_weight_max': 1.0, 'd_max': 160.0, 'nsyn_min': 3, 'nsyn_max': 7},\n",
    "              syn_weight=-3.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='instanteneousInh.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "### Generating I-to-E connections\n",
    "net.add_edges(source={'ei': 'i'}, target={'ei': 'e', 'pop_name': 'Scnn1a'},\n",
    "              connection_rule=distance_connector,\n",
    "              connection_params={'d_weight_min': 0.0, 'd_weight_max': 1.0, 'd_max': 160.0, 'nsyn_min': 3, 'nsyn_max': 7},\n",
    "              syn_weight=-3.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='InhToExc.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "net.add_edges(source={'ei': 'i'}, target={'ei': 'e', 'pop_name': 'LIF_exc'},\n",
    "              connection_rule=distance_connector,\n",
    "              connection_params={'d_weight_min': 0.0, 'd_weight_max': 1.0, 'd_max': 160.0, 'nsyn_min': 3, 'nsyn_max': 7},\n",
    "              syn_weight=-3.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='instanteneousInh.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "### Generating E-to-I connections\n",
    "net.add_edges(source={'ei': 'e'}, target={'pop_name': 'PV'},\n",
    "              connection_rule=distance_connector,\n",
    "              connection_params={'d_weight_min': 0.0, 'd_weight_max': 0.26, 'd_max': 300.0, 'nsyn_min': 3, 'nsyn_max': 7},\n",
    "              syn_weight=3.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='ExcToInh.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "\n",
    "net.add_edges(source={'ei': 'e'}, target={'pop_name': 'LIF_inh'},\n",
    "              connection_rule=distance_connector,\n",
    "              connection_params={'d_weight_min': 0.0, 'd_weight_max': 0.26, 'd_max': 300.0, 'nsyn_min': 3, 'nsyn_max': 7},\n",
    "              syn_weight=3.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='instanteneousExc.json',\n",
    "              model_template='static_synapse')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "net.build()\n",
    "net.save_nodes(output_dir='sim_ch05/network')\n",
    "net.save_edges(output_dir='sim_ch05/network')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Building external network\n",
    "\n",
    "Next we want to create an external network of \"virtual cells\" with spike-trains that will synapse onto our V1 cells and drive activity. We will call this external network \"LGN\" and contains 500 excitatory cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "lgn = NetworkBuilder('LGN')\n",
    "lgn.add_nodes(N=500, pop_name='tON', potential='exc', model_type='virtual')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use a special function for setting the number of synapses between the LGN --> V1 cells. The select_source_cells function will be called during the build process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bmtk.builder.connection_map.ConnectionMap at 0x7ffbbc32f588>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "def select_source_cells(sources, target, nsources_min=10, nsources_max=30, nsyns_min=3, nsyns_max=12):\n",
    "    total_sources = len(sources)\n",
    "    nsources = np.random.randint(nsources_min, nsources_max)\n",
    "    selected_sources = np.random.choice(total_sources, nsources, replace=False)\n",
    "    syns = np.zeros(total_sources)\n",
    "    syns[selected_sources] = np.random.randint(nsyns_min, nsyns_max, size=nsources)\n",
    "    return syns\n",
    "\n",
    "lgn.add_edges(source=lgn.nodes(), target=net.nodes(pop_name='Scnn1a'),\n",
    "              iterator='all_to_one',\n",
    "              connection_rule=select_source_cells,\n",
    "              connection_params={'nsources_min': 10, 'nsources_max': 25},\n",
    "              syn_weight=15.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='ExcToExc.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "lgn.add_edges(source=lgn.nodes(), target=net.nodes(pop_name='PV1'),\n",
    "              connection_rule=select_source_cells,\n",
    "              connection_params={'nsources_min': 15, 'nsources_max': 30},\n",
    "              iterator='all_to_one',\n",
    "              syn_weight=5.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='ExcToInh.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "lgn.add_edges(source=lgn.nodes(),  target=net.nodes(pop_name='LIF_exc'),\n",
    "              connection_rule=select_source_cells,\n",
    "              connection_params={'nsources_min': 10, 'nsources_max': 25},\n",
    "              iterator='all_to_one',\n",
    "              syn_weight= 10.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='instanteneousExc.json',\n",
    "              model_template='static_synapse')\n",
    "\n",
    "lgn.add_edges(source=lgn.nodes(),  target=net.nodes(pop_name='LIF_inh'),\n",
    "              connection_rule=select_source_cells,\n",
    "              connection_params={'nsources_min': 15, 'nsources_max': 30},\n",
    "              iterator='all_to_one',\n",
    "              syn_weight=10.0,\n",
    "              delay=2.0,\n",
    "              dynamics_params='instanteneousExc.json',\n",
    "              model_template='static_synapse')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we build and save our lgn network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "lgn.build()\n",
    "lgn.save_nodes(output_dir='sim_ch05/network')\n",
    "lgn.save_edges(output_dir='sim_ch05/network')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Setting up PointNet Environment\n",
    "\n",
    "### Directory Structure\n",
    "\n",
    "Before running a simulation, we will need to create the runtime environment, including parameter files, run-script and configuration files. If using the tutorial these files will already be in place. Otherwise we can use a command-line:\n",
    "```bash\n",
    "$ python -m bmtk.utils.sim_setup        \\\n",
    "    --network sim_ch05/network/         \\\n",
    "    --include-examples                  \\\n",
    "    --tstop 3000.0                      \\\n",
    "    pointnet sim_ch05/\n",
    "\n",
    "```\n",
    "\n",
    "or"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bmtk.utils.sim_setup import build_env_pointnet\n",
    "\n",
    "build_env_pointnet(base_dir='sim_ch05',      \n",
    "                   network_dir='sim_ch05/network',\n",
    "                   tstop=3000.0, \n",
    "                   dt=0.01,\n",
    "                   include_examples=True,         # Copies components files\n",
    "                  )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The network files are written to **circuit_config.json** and the simulation parameters are set in **simulation_config**. The simulation time is set to run for 3000.0 ms (tstop). We also specify a membrane-report to record V_m property of 4 cells (gids 0, 80, 100, 300 - one from each cell-type). In general, all the parameters needed to setup and start a simulation are found in the config files, and adjusting network/simulation conditions can be done by editing these json files in a text editor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### lgn input\n",
    "\n",
    "We need to provide our LGN external network cells with spike-trains so they can activate our recurrent network. Previously we showed how to do this by generating csv files. We can also use NWB files, which are a common format for saving electrophysiological data in neuroscience.\n",
    "\n",
    "We can use any NWB file generated experimentally or computationally, but for this example we will use a preexsting one. First download the file:\n",
    "```bash\n",
    " $ cd sim_ch05\n",
    " $ wget https://github.com/AllenInstitute/bmtk/raw/develop/docs/examples/spikes_inputs/lgn_spikes.h5\n",
    "```\n",
    "Then we must edit the simulation_config.json file to tell the simulator to find the nwb file and which network to associate it with.\n",
    "\n",
    "```json\n",
    "{\n",
    "  \"inputs\": {\n",
    "    \"LGN_spikes\": {\n",
    "      \"input_type\": \"spikes\",\n",
    "      \"module\": \"h5\",\n",
    "      \"input_file\": \"$BASE_DIR/lgn_spikes.h5\",\n",
    "      \"node_set\": \"LGN\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Running the simulation\n",
    "\n",
    "The call to sim_setup created a file run_pointnet.py which we can run directly in a command line:\n",
    "```bash\n",
    "$ python run_pointnet.py config.json\n",
    "```\n",
    "or if you have mpi setup:\n",
    "\n",
    "```bash\n",
    "$ mpirun -np $NCORES python run_pointnet.py config.json\n",
    "```\n",
    "\n",
    "Or we can run it directly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-08-25 15:34:08,266 [INFO] Created log file\n",
      "2020-08-25 15:34:08,307 [INFO] Batch processing nodes for LGN/0.\n",
      "2020-08-25 15:34:08,361 [INFO] Batch processing nodes for V1/0.\n",
      "2020-08-25 15:34:08,449 [INFO] Setting up output directory\n",
      "2020-08-25 15:34:08,450 [INFO] Building cells.\n",
      "2020-08-25 15:34:08,495 [INFO] Building recurrent connections\n",
      "2020-08-25 15:34:08,628 [INFO] Build virtual cell stimulations for LGN_spikes\n",
      "2020-08-25 15:34:53,999 [INFO] Network created.\n",
      "2020-08-25 15:34:54,463 [INFO] Starting Simulation\n",
      "2020-08-25 15:35:01,445 [INFO] Simulation finished, finalizing results.\n",
      "2020-08-25 15:35:02,240 [INFO] Done.\n"
     ]
    }
   ],
   "source": [
    "from bmtk.simulator import pointnet\n",
    "\n",
    "configure = pointnet.Config.from_json('sim_ch05/simulation_config.json')\n",
    "configure.build_env()\n",
    "network = pointnet.PointNetwork.from_config(configure)\n",
    "sim = pointnet.PointSimulator.from_config(configure, network)\n",
    "sim.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Analyzing results\n",
    "\n",
    "Results of the simulation, as specified in the config, are saved into the output directory. Using the analyzer functions, we can do things like plot the raster plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 21064 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from bmtk.analyzer.spike_trains import plot_raster, plot_rates\n",
    "\n",
    "plot_raster(config_file='sim_ch05/simulation_config.json', group_by='pop_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we can plot the rates of the different populations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 14780 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_rates(config_file='sim_ch05/simulation_config.json', group_by='pop_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In our simulation_config.json in the reports section, we can see we also record the V_m (i.e membrane potential) of a select sample of cells. By default these files are written to an hdf5 file with the same name as the report (membrane_potential.h5), and we can use the analyzer to show the time course of some of these cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 5. Additional Information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Recording membrane potential  (and other cell properties).\n",
    "\n",
    "By default bmtk will automatically record and save the spike-trains of our (non-virtual) cells. Sometimes it can be useful to record other properties besides spikes, like the membrane voltage traces, of all or a subsection of the cells. To do so we only need to open the _simulation_config.json_ with our prefered text editor and insert the following section:\n",
    "\n",
    "```json\n",
    "{\n",
    "  \"reports\": {\n",
    "    \"membrane_potential\": {\n",
    "      \"cells\": {\n",
    "        \"population\": \"V1\", \n",
    "        \"node_id\": [0, 20, 60, 80, 100]\n",
    "      },\n",
    "      \"variable_name\": \"V_m\",\n",
    "      \"module\": \"multimeter_report\",\n",
    "      \"sections\": \"soma\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "```\n",
    "\n",
    "This will tell bmtk to record the \"V_m\" variable for selected cells with ids 0, 20, 60, 80, 100 - and save the results in a file _output/membrane_potential.h5_. After re-running the simulation the file will be generated, [according to the SONATA format](https://github.com/AllenInstitute/sonata/blob/master/docs/SONATA_DEVELOPER_GUIDE.md#frame-oriented-node-element-recordings), which we can read using hdf5 or bmtk like below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 70896 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from bmtk.analyzer.compartment import plot_traces\n",
    "\n",
    "_ = plot_traces(config_file='sim_ch05/simulation_config.json', group_by='pop_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perturbation simulations\n",
    "\n",
    "A common use case is to mimick a network where the firing rate of select cells are being depressed or stimulated using current clamps or optogenetic methods. The best way to do this is by applying a current clamp in a similar manner used in [bionet](http://localhost:32000/notebooks/Workspace/bmtk/docs/tutorial/01_single_cell_clamped.ipynb).\n",
    "\n",
    "The point_120cells network in the bmtk examples consists of 120 recurrently connected excitatory and inhibitory cells with randomized inputs. Normally the results without any type of perturbations will look as follows:\n",
    "<img src=\"_static/_tutorial_images/raster_120cells_orig.png\">\n",
    "\n",
    "To artifically excite some of the cells we can open the config.json file and add the following section to the \"inputs\":\n",
    "```json\n",
    "{\n",
    "  \"exc_perturbation\": {\n",
    "    \"input_type\": \"current_clamp\",\n",
    "    \"module\": \"IClamp\",\n",
    "    \"node_set\": {\n",
    "      \"population\": \"cortex\",\n",
    "      \"node_ids\": [20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39]\n",
    "    },\n",
    "    \"amp\": 230.0,\n",
    "    \"delay\": 1.0,\n",
    "    \"duration\": 3000.0\n",
    "  }\n",
    "}\n",
    "```\n",
    "\n",
    "This will apply a large step current for the duration of the simulation for cells 20 through 39. In this case we knew the exact node_ids of the cells we want to excite. However it is also possible to get a subset using a specific property, for example if we wanted to only select inhibitory L2/3 cells:\n",
    "```json\n",
    "{\n",
    "  \"node_set\": {\n",
    "    \"population\": \"cortex\",\n",
    "    \"ei\": \"i\",\n",
    "    \"location\": \"L2/3\"\n",
    "  }\n",
    "}\n",
    "```\n",
    "\n",
    "To simulate depression of cells 40 to 49 we will add another current_clamp but with a large negative current:\n",
    "```json\n",
    "{\n",
    "  \"inh_perturbation\": {\n",
    "    \"input_type\": \"current_clamp\",\n",
    "    \"module\": \"IClamp\",\n",
    "    \"node_set\": {\n",
    "      \"population\": \"cortex\",\n",
    "      \"node_ids\": [40, 41, 42, 43, 44, 45, 46, 47, 48, 49]\n",
    "    },\n",
    "    \"amp\": -230.0,\n",
    "    \"delay\": 1.0,\n",
    "    \"duration\": 3000.0\n",
    "  }\n",
    "}\n",
    "```\n",
    "\n",
    "After we rerun the simulation:\n",
    "<img src=\"_static/_tutorial_images/raster_120cells_perturbed.png\">\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
