Y. G. Drevs, A. S.Migalev

This review is devoted to investigation, modeling and implementation of spike-timing-dependent plasticity. Spike-timing dependent plasticity (STDP) is a type of synaptic plastisticity where change in synaptic strength depends on time order of pre- and postsynaptic spike. It is a subject of many neurophysiological investigations. The properties of STDP depend on type of synaptic transmission, synapse location and type of the targeting cell. Works on experimental research of this type of plasticity on excitatory and inhibitory synapses, using complex stimuli, are reviewed. Theoretical studies investigate its role in working neural network, properties that STDP add to a model of neural network, compute optimal function of STDP using various hypotheses on tasks of this type of synaptical plasticity in neural networks. Some investigations is devoted to reward modulated STDP and how it is related to theory of reinforcement learning. They offer a connection of cell mechanisms of synaptic plasticity with animal learning. Learning algorithms of neural networks models on basis of STDP are reviewed, algorithms that take into account complex spike trains on neuron input. An important property of this learning algorithms is that they use information about activity of two cells for synaptic strength change. This property allows to implement learning algorithm in neural network model with large amount of neurons. Some studies are devoted to methods for modeling neural networks, using STDP on basis of analog and digital computing systems. Implementation of neuron models with STDP on analog VLSI is made using various circuits on chips or using external units for synaptic strength change. STDP is used as an learning algorithm in robots navigating systems, for synchrony search in spike trains and in many others technical applications

1. Никколс Д., Мартин Р., Валлас Б., Пол Ф. От нейрона к мозгу / Пер. С англ. П. М. Балабана, А. В. Галкина, Р. А. Гиниатуллина, Р. Н. Хазипова, Л. С. Хируга. М.: Едиториал УРСС. 2003. 672 с.
2. Purves D., Augustine G.J., Fitzpatrick D., Hall W.C. Anthony-Samuel Lamantia (Editor), McNamara J. O., Williams S. M.  Neuroscience // Sinauer Associates. June 2004. 773 pp.
3. Скребицкий В.Г. Синаптическая пластичность как проблема в нейрофизиологии // ВЕСТНИК РФФИ. Декабрь 2004. № 4(38).
4. Bi G-Q., Poo M.M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type // J. Neurosci. 1998 Dec. 15; 18(24) Р. 10464-72.
5. Abbott L.F., Gerstner W. Homeostasis and learning through spike-timing dependent plasticity // In D. Hansel, C. Chow, B. Gutkin, and C. Meunier,  editors, Methods and Models in Neurophysics, 2004.
6. Dan Y., Poo M.M. Spike timing-dependent plasticity: from synapse to perception // Physiol Rev. 2006 Jul. V. 86. № 3. Р. 1033-48
7. Song S., Miller K.D., Abbott L.F. Competitive Hebbian learning through spike-timing-dependent synaptic plasticity // Nat Neurosci. 2000. V. 3. № 9. Р. 919-26.
8. Froemke R.C., Dan Y. Spike-timing-dependent synaptic modification induced by natural spike trains // Nature. 2002 Mar. V. 28; 416(6879). P. 433-8.
9. Nelson S.B., Sjöström P.J., Turrigiano G.G. Rate and timing in cortical synaptic plasticity // Philos Trans R Soc Lond B Biol Sci. 2002 Dec. V. 29; 357(1428). P. 1851-7.
10. Kampa B.M., Letzkus J.J., Stuart G.J.  Requirement of dendritic calcium spikes for induction of spike-timing-dependent synaptic plasticity // J. Physiol. 2006 Jul. V. 1;574(Pt. 1). P. 283-90. Epub. 2006 May 4.
11. Kampa B.M., Stuart G.J. Calcium spikes in basal dendrites of layer 5 pyramidal neurons during action potential bursts // J. Neurosci. 2006 Jul. V. 12; 26(28). P. 7424-32.
12. Kobayashi K, Poo M.M. Spike train timing-dependent associative modification of hippocampal CA3 recurrent synapses by mossy fibers // Neuron. 2004 Feb. V. 5; 41(3). P. 445-54.
13. Froemke R.C., Tsay I.A., Raad M., Long J.D., Dan Y. Contribution of individual spikes in burst-induced long-term synaptic modification // J. Neurophysiol. 2006 Marh. V.95(3). P. 1620-9. Epub 2005 Nov. 30.
14. Wang H.X., Gerkin R.C., Nauen D.W., Bi G-Q. Coactivation and timing-dependent integration of synaptic potentiation and depression // Nat. Neurosci. 2005 Feb. V. 8(2). P. 187-93. Epub 2005 Jan 16.
15. Haas J.S., Nowotny T., Abarbanel H.D. Spike-timing-dependent plasticity of inhibitory synapses in the entorhinal cortex // J. Neurophysiol. 2006 Dec. V. 96(6). P. 3305-13. Epub 2006 Aug 23.
16. Tzounopoulos T., Rubio M.E., Keen J.E., Trussell L.O. Coactivation of pre- and postsynaptic signaling mechanisms determines cell-specific spike-timing-dependent plasticity // Neuron. 2007 Apr. V. 19; 54(2). P. 291-301.
17. Rumsey C.C., Abbott L.F. Synaptic democracy in active dendrites // J. Neurophysiol. 2006 Nov. V. 96(5). P. 2307-18. Epub 2006 Jul 12.
18. Letzkus J.J., Kampa B.M., Stuart G.J. Learning rules for spike timing-dependent plasticity depend on dendritic synapse location // J. Neurosci. 2006 Oct. V. 11; 26(41). P. 10420-9.
19. Lu J.T., Li C.Y., Zhao J.P., Poo M.M. Zhang XH.  Spike-timing-dependent plasticity of neocortical excitatory synapses on inhibitory interneurons depends on target cell type // J. Neurosci. 2007 Sep. V. 5; 27(36). P. 9711-20.
20. Izhikevich E.M., Gally J.A., Edelman G.M. Spike-timing dynamics of neuronal groups // Cereb Cortex. 2004 Aug. V. 14(8). P. 933-44. Epub 2004 May 13.
21. Masquelier T., Guyonneau R., Thorpe S.J. Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains // Cereb Cortex. 2004 Aug. V. 14(8). P. 933-44. Epub 2004 May 13.
22. Young J.M., Waleszczyk W.J., Wang C., Calford M.B., Dreher B., Obermayer K. Cortical reorganization consistent with spike timing-but not correlation-dependent plasticity // Nat. Neurosci. 2007 Jul. V. 10(7). P. 887-95. Epub 2007 May 27.
23. Nowotny T., Zhigulin V.P., Selverston A.I., Abarbanel H.D., Rabinovich M.I. Enhancement of synchronization in a hybrid neural circuit by spike-timing dependent plasticity // J. Neurosci. 2003 Oct. 29;23(30). P. 9776-85.
24. Masuda N., Kori H. Formation of feedforward networks and frequency synchrony by spike-timing-dependent plasticity // J. Comput Neurosci. 2007 Jun.; V. 22(3). P. 327-45. Epub 2007 March 28.
25. Hopfield J.J., Brody C.D. Learning rules and network repair in spike-timing-based computation networks // Proc. Natl Acad. Sci. USA. 2004 Jan. V. 6;101(1). P. 337-42. Epub 2003 Dec. 23.
26. Toyoizumi T., Pfister J-P., Aihara K., Gerstner W. Spike-timing dependent plasticity and mutual information maximization for a spiking neuron model // Advances in Neural Information Processing Systems 17, 2005, p. 1409-1416 MIT Press, 2005.
27. Gerstner W., Kistler W.M. Spiking Neuron Models: Single Neurons, Populations, Plasticity // Cambridge University Press; 1 edition (August 15, 2002), 400 pp.  
28. Bohte S.M., Mozer M.C. Reducing spike train variability:a computational theory of spike-timing dependent plastisity // Proceedings of advances in neural information processing (NIPS)17 Vancouver, Canada, 2004
29. Pfister J.P., Toyoizumi T., Barber D., Gerstner W. Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning // Neural Comput. 2006 Jun. V. 18(6). P. 1318-48.
30. Carr C.E., Konishi M. A circuit for detection of interaural time differences in the brain stem of the barn owl // J. Neurosci. 1990 Oct; V. 10(10). P. 3227-46.
31. Fetz E.E., Baker M.A. Operantly conditioned patterns on precentral unit activity and correlated responses in adjacent cells and contralateral muscles // J. Neurophysiol. 1973 March. V. 36(2). P. 179-204.
32. Fetz E.E. Volitional control of neural activity: implications for brain-computer interfaces // J. Physiol. 2007 March. V. 15; 579(Pt 3). V. 571-9. Epub 2007 Jan 18.
33. Chapin J.K., Moxon K.A., Markowitz R.S., Nicolelis M.A.  Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex // Nat. Neurosci. 1999 Jul. V. 2(7). P. 664-70.
34. Carmena J.M., Lebedev M.A., Crist R.E., O'Doherty J.E., Santucci D.M., Dimitrov D.F., Patil P.G., Henriquez C.S., Nicolelis M.A. Learning to control a brain-machine interface for reaching and grasping by primates // PLoS Biol. 2003 Nov. V. 1(2). P. E42. Epub 2003 Oct. 13.
35. Taylor D.M., Tillery S.I., Schwartz A.B. Direct cortical control of 3D neuroprosthetic devices // Science. 2002 Jun. V.  7; 296(5574). P. 1829-32.
36. Velliste M., Perel S., Spalding M.C., Whitford A.S., Schwartz A.B. Cortical control of a prosthetic arm for self-feeding // Nature. 2008 Jun. V. 19; 453(7198). P. 1098-101. Epub 2008 May 28.
37. Legenstein R., Pecevski D., Maass W. Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity // Proceedings of advances in neural information processing (NIPS) 2007, 3-8 Dec 2007, Vancouver, Canada.
38. Izhikevich E.M. Solving the distal reward problem through linkage of STDP and dopamine signaling // Cereb Cortex. 2007 Oct. V. 17(10). P. 2443-52. Epub 2007 Jan. 13.
39. Legenstein R., Pecevski D., Maass W. Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity // Proceedings of advances in neural information processing 2007 (NIPS 2007). December 5, 2007
40. Schultz W. Predictive reward signal of dopamine neurons // J. Neurophysiol. 1998 Jul. V. 80(1). P. 1-27.
41. Bartlett P.L., Baxter J. Estimation and approximation bounds for gradient-based reinforcement learning // Proceedings of the Thirteenth Annual Conference on Computational Learning Theory. 2000. P. 133-141.
42. Baxter J., Bartlett H.L. Infinite-horizon policy-gradient estimation // Journal of Artificial Intelligence Research. 2001. 15. P. 319-350.
43. Florian R.V. Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity // Neural Computation. 2007. 19(6). P. 1468-1502.
44. Cai Y., Gavornik J.P., Cooper L.N., Yeung L.C., Shouval H.Z. Effect of stochastic synaptic and dendritic dynamics on synaptic plasticity in visual cortex and hippocampus // J. Neurophysiol. 2007 Jan.; V. 97(1). P. 375-86. Epub 2006. Oct. 11.
45. Pfister J.P., Gerstner W. Triplets of spikes in a model of spike timing-dependent plasticity // J. Neurosci. 2006 Sep. V. 20; 26(38). P. 9673-82.
46. Liu X., Wang Y., Zhou C. A new formula of synaptic plasticity based on STDP and its implications // International Conference on Neural Networks and Brain, 2005 (ICNN&B '05).
47. Standage D., Trappenberg T. The trouble width weight-dependent STDP // Proceedings of International Joint Conference on Neural Networks, Orlando, Florida, USA, August 12-17, 2007.
48. Dockendorf K.P., DeMarse T.B. Amplitude and spike timing dependent plasticity // Proceedings of International Joint Conference on Neural Networks, Orlando, Florida, USA, August 12-17, 2007.
49. Schemmel J., Meier K., Mueller E. A new VLSI model of neural microcircuits including spike time dependent plasticity // Proceedings of International Joint Conference on Neural Networks, Budapest, Hungary, 25-29 July, 2004.
50. Mehrtash N., Jung D., Hellmich H.H., Schoenauer T., Lu V.T., Klar H. Synaptic plasticity in spiking neural networks (SP2INN): a system approach //  IEEE Transactions on  Neural Networks. Sept. 2003. V. 14, Is. 5. P. 980-992.
51. Schaefer M., Schoenauer T., Wolff C.,  Hartmann G., Klar H., Ruckert U. Simulation of spiking neural networks - architectures and implementations // Elsevier,  Neurocomputing. October 2002. V. 48, № 1. Р. 647-679(33).
52. Grassmann C., Schoenauer T., Wolff C. PCNN neurocomputers - event driven and parallel architectures //  European Symposium on Artificial Neural Networks (ESANN'2002). Belgium, 24-26 April 2002. Р. 331-336.
53. Indiveri, G., Chicca, E., Douglas, R. A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity // Transactions on Neural Networks, IEEE. Volume 17, Issue 1, Jan. 2006 Page(s): 211 - 221.
54. Vogelstein R.J. Mallik U., Vogelstein J.T., Cauwenberghs G. Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses // IEEE Transactions on Neural Networks. Jan. 2007. V. 18. Is. 1. Р. 253-265.
55. Renaud S., Tomas J., Bornat Y., Daouzli A., Saighi S.  Neuromimetic ICs with analog cores: an alternative for simulating spiking neural networks // IEEE International Symposium on  Circuits and Systems (ISCAS 2007), New Orleans, LA,  2007. 27-30 May 2007.
56. Arena P., Fortuna L., Frasca M., Patane L., Sala C. Integrating high-level sensor features via STDP for bio-inspired navigation // IEEE International Symposium on Circuits and Systems, 2007. ISCAS 2007. New Orleans, LA. May 2007. Р. 609-612.
57. Nielsen, J., Lund, H.H. Spiking neural building block robot with Hebbian learning // International Conference on Intelligent Robots and Systems, 2003. (IROS 2003). Oct. 2003. V. 2. Р. 1363-1369.
58. Mokhtar M., Halliday D.M., Tyrrel A. M. Autonomus navigational controller inspired by the hyppocampus // Proceedings of international joint conference on neural networks , Oriando, Florida, USA, August 12-17, 2007.
59. Tovar G.M., Fukuda E.S., Asai T., Hirose T., Amemiya Y. Neuromorphic CMOS circuits implementing a novel neural segmentation model based on symmetric STDP learning // International Joint Conference on Neural Networks, 2007. (IJCNN 2007). Orlando, Florida. Aug. 2007. Р. 897-901.
60. Koickal T.J. Hamilton A., Tan S.L., Covington J.A., Gardner J.W., Pearce T.C. Analog VLSI Circuit Implementation of an Adaptive Neuromorphic Olfaction Chip //  IEEE Transactions on  Circuits and Systems I: Regular Papers, [ IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications]. Jan. 2007. V. 54. Is. 1. Р. 60-73.
61. Cameron, K. Murray, A. Collins, S. Spike timing dependent adaptation for mismatch compensation // International Symposium on Circuits and Systms, 2006. (ISCAS 2006). 21-24 May 2006. Рage(s): 4. Р. 1223-226.
62. Cameron K., Boonsobhak V., Murray A., Renshaw D. Spike timing dependent plasticity (STDP) can ameliorate process variations in neuromorphic VLSI // IEEE Transactions on Neural Networks. Nov. 2005. V. 16. Is. 6. Р. 1626-1637.
63. Yang Z., Murray A., Worgotter F., Cameron K., Boonsobhak V. A neuromorphic depth-from-motion vision model with STDP adaptation // IEEE Transactions on Neural Networks. March 2006. V. 17. Is. 2. P. 482-495.
64. Gupta A., Long L.N. Character recognition using spiking neural networks // International Joint Conference on Neural Networks, 2007(IJCNN 2007) Orlando, FL,. Aug. 2007. Р. 53-58.
65. Saeki K., Hayashi Y., Sekine Y. Extraction of Phase Information Buried in Fluctuation of a Pulse-type Hardware Neuron Model Using STDP //  International Joint Conference on Neural Networks, 2006. (IJCNN apos;06) Vancouver, BC. Volume, Issue, 0-0 0 Page(s):1505-1510.
66. Bofill-i-Petit A., Murray A.F. Synchrony detection and amplification by silicon neurons with STDP synapses // IEEE Transactions on neural networks. 2004. V. 15. № 5.
67. Maass W., Bishop C.M. Pulsed neural networks // The MIT Press (March 1, 2001) 407 pp.