19/08/03 20:03:23.00 ahZr93IIX
Robustness and fault tolerance
HTM Very high
HMMs No
LSTM Yes
Detailed mapping to neuroscience
HTM Yes
HMMs No
LSTM No
Probabilistic model
HTM No
HMMs Yes
LSTM No
S1 Table, Comparison of Common Sequence Memory Algorithms Table comparing two common sequence memory algorithms (HMM and LSTM) to proposed model (HTM).
* Although weight updated rules are local, LSTMs require computing a global error signal that is then back propagated.
20
709:YAMAGUTIseisei
19/08/11 18:01:19.35 TqTByUJCx
Universal Transformers URLリンク(arxiv-vanity.com)
References
[1] Karim Ahmed, Nitish Shirish Keskar, and Richard Socher.
Weighted transformer network for machine translation.
arXiv preprint arXiv:1711.02132, 2017.
[2] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton.
Layer normalization. arXiv preprint
arXiv:1607.06450, 2016.
[3] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio.
Neural machine translation by jointly learning to align and translate.
CoRR, abs/1409.0473, 2014.
[4] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio.
Learning phrase representations using RNN encoder-decoder for statistical machine translation.
CoRR, abs/1406.1078, 2014.
[5] Francois Chollet.
Xception: Deep learning with depthwise separable convolutions.
arXiv preprint arXiv:1610.02357, 2016.
[6] Zewei Chu, Hai Wang, Kevin Gimpel, and David McAllester.
Broad context language modeling as reading comprehension.
In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, volume 2, pages 52–57, 2017.
[7] Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov.
Neural models for reasoning over multiple mentions using coreference.
arXiv preprint arXiv:1804.05922, 2018.
[8] Bhuwan Dhingra, Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov.
Linguistic knowledge as memory for recurrent neural networks.
arXiv preprint arXiv:1703.02620, 2017.
[9] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin.
Convolutional sequence to sequence learning.
CoRR, abs/1705.03122, 2017.
[10] Edouard Grave, Armand Joulin, and Nicolas Usunier.
Improving neural language models with a continuous cache.
arXiv preprint arXiv:1612.04426, 2016.
710:YAMAGUTIseisei
19/08/11 18:02:07.89 TqTByUJCx
[11] Alex Graves.
Generating sequences with recurrent neural networks.
CoRR, abs/1308.0850, 2013.
[12] Alex Graves.
Adaptive computation time for recurrent neural networks.
arXiv preprint arXiv:1603.08983, 2016.
[13] Alex Graves, Greg Wayne, and Ivo Danihelka.
Neural turing machines.
CoRR, abs/1410.5401, 2014.
[14] Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun.
Tracking the world state with recurrent entity networks.
arXiv preprint arXiv:1612.03969, 2016.
[15] Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, and Jürgen Schmidhuber.
Gradient flow in recurrent nets: the difficulty of learning long-term dependencies.
A Field Guide to Dynamical Recurrent Neural Networks, 2003.
[16] A. Joulin and T. Mikolov.
Inferring algorithmic patterns with stack-augmented recurrent nets.
In Advances in Neural Information Processing Systems, (NIPS), 2015.
[17] Łukasz Kaiser and Ilya Sutskever.
Neural GPUs learn algorithms.
In International Conference on Learning Representations (ICLR), 2016.
[18] Nikita Kitaev and Dan Klein.
Constituency parsing with a self-attentive encoder.
In Proceedings of ACL’18, 2018.
[19] Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio.
A structured self-attentive sentence embedding.
arXiv preprint arXiv:1703.03130, 2017.
[20] Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg.
Assessing the ability of lstms to learn syntax-sensitive dependencies.
Transactions of the Association of Computational Linguistics, 4(1):521–535, 2016.
711:YAMAGUTIseisei
19/08/11 18:05:21.30 TqTByUJCx
[21] Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernandez.
The lambada dataset: Word prediction requiring a broad discourse context.
In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1525–1534, 2016.
[22] Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit.
A decomposable attention model.
In Empirical Methods in Natural Language Processing, 2016.
[23] Jack Rae, Jonathan J Hunt, Ivo Danihelka, Timothy Harley, Andrew W Senior, Gregory Wayne, Alex Graves, and Tim Lillicrap.
Scaling memory-augmented neural networks with sparse reads and writes.
In Advances in Neural Information Processing Systems, pages 3621–3629, 2016.
[24] Minjoon Seo, Sewon Min, Ali Farhadi, and Hannaneh Hajishirzi.
Query-reduction networks for question answering.
arXiv preprint arXiv:1606.04582, 2016.
[25] Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.
Dropout: a simple way to prevent neural networks from overfitting.
Journal of Machine Learning Research, 15(1):1929–1958, 2014.
[26] Sainbayar Sukhbaatar, arthur szlam, Jason Weston, and Rob Fergus.
End-to-end memory networks.
In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 2440–2448. Curran Associates, Inc., 2015.
[27] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le.
Sequence to sequence learning with neural networks.
In Advances in Neural Information Processing Systems, pages 3104–3112, 2014.
[28] Ke Tran, Arianna Bisazza, and Christof Monz.
The importance of being recurrent for modeling hierarchical structure.
In Proceedings of NAACL’18, 2018.
712:YAMAGUTIseisei
19/08/11 18:06:08.63 TqTByUJCx
[29] Ashish Vaswani, Samy Bengio, Eugene Brevdo, Francois Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Łukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, and Jakob Uszkoreit.
Tensor2tensor for neural machine translation.
CoRR, abs/1803.07416, 2018.
[30] Ashish Vaswani, Niki Parmar, Jakob Uszkoreit, Noam Shazeer, and Lukasz Kaiser.
Image transformer
, 2018.
[31] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
CoRR, 2017.
[32] Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov.
Towards ai-complete question answering: A set of prerequisite toy tasks.
arXiv preprint arXiv:1502.05698, 2015.
[33] Dani Yogatama, Yishu Miao, Gabor Melis, Wang Ling, Adhiguna Kuncoro, Chris Dyer, and Phil Blunsom.
Memory architectures in recurrent neural network language models.
In International Conference on Learning Representations, 2018.
[34] Wojciech Zaremba and Ilya Sutskever.
Learning to execute.
CoRR, abs/1410.4615, 2015.
713:YAMAGUTIseisei
19/08/11 18:06:34.23 TqTByUJCx
Figure 4: The Universal Transformer with position and step embeddings as well as dropout and layer normalization.
Appendix B
bAbI Detailed Results
Best seed run for each task (out of 10 runs)
Task id
10K
1K
train single , train joint , train single , train joint
1 0.0 0.0 0.0 0.0
2 0.0 0.0 0.0 0.5
3 0.4 1.2 3.7 5.4
4 0.0 0.0 0.0 0.0
5 0.0 0.0 0.0 0.5
6 0.0 0.0 0.0 0.5
7 0.0 0.0 0.0 3.2
8 0.0 0.0 0.0 1.6
9 0.0 0.0 0.0 0.2
10 0.0 0.0 0.0 0.4
11 0.0 0.0 0.0 0.1
12 0.0 0.0 0.0 0.0
13 0.0 0.0 0.0 0.6
14 0.0 0.0 0.0 3.8
15 0.0 0.0 0.0 5.9
16 0.4 1.2 5.8 15.4
17 0.6 0.2 32.1 43.2
18 0.0 0.0 0.0 4.1
19 2.8 3.1 47.2 69.11
20 0.0 0.0 2.4 2.4
avg err 0.21 0.29 4.56 7.85
failed 0 0 3 5
Average (±var) over all seeds (for 10 runs)
Task id 10K 1K
714:YAMAGUTIseisei
19/08/11 18:07:04.04 TqTByUJCx
train single
train joint
train single
train joint
1 0.0 ±0.0 0.0 ±0.0 0.2 ±0.3 0.1 ±0.2
2 0.2 ±0.4 1.7 ±2.6 3.2 ±4.1 4.3 ±11.6
3 1.8 ±1.8 4.6 ±7.3 9.1 ±12.7 14.3 ±18.1
4 0.1 ±0.1 0.2 ±0.1 0.3 ±0.3 0.4 ±0.6
5 0.2 ±0.3 0.8 ±0.5 1.1 ±1.3 4.3 ±5.6
6 0.1 ±0.2 0.1 ±0.2 1.2 ±2.1 0.8 ±0.4
7 0.3 ±0.5 1.1 ±1.5 0.0 ±0.0 4.1 ±2.9
8 0.3 ±0.2 0.5 ±1.1 0.1 ±0.2 3.9 ±4.2
9 0.0 ±0.0 0.0 ±0.0 0.1 ±0.1 0.3 ±0.3
10 0.1 ±0.2 0.5 ±0.4 0.7 ±0.8 1.3 ±1.6
11 0.0 ±0.0 0.1 ±0.1 0.4 ±0.8 0.3 ±0.9
12 0.2 ±0.1 0.4 ±0.4 0.6 ±0.9 0.3 ±0.4
13 0.2 ±0.5 0.3 ±0.4 0.8 ±0.9 1.1 ±0.9
14 1.8 ±2.6 1.3 ±1.6 0.1 ±0.2 4.7 ±5.2
15 2.1 ±3.4 1.6 ±2.8 0.3 ±0.5 10.3 ±8.6
16 1.9 ±2.2 0.9 ±1.3 9.1 ±8.1 34.1 ±22.8
17 1.6 ±0.8 1.4 ±3.4 44.7 ±16.6 51.1 ±12.3
18 0.3 ±0.4 0.7 ±1.4 2.3 ±3.6 12.8 ±9.0
19 3.4 ±4.0 6.1 ±7.3 50.2 ±8.4 73.1 ±23.9
20 0.0 ±0.0 0.0 ±0.0 3.2 ±2.5 2.6 ±2.8
avg 0.73 ±0.89 1.12 ±1.62 6.39 ±3.22 11.21 ±6.62
715:YAMAGUTIseisei
19/08/11 18:07:28.73 TqTByUJCx
Appendix C
bAbI Attention Visualization
We present visualization of the attention distributions on bAbI tasks for a couple of examples.
The visualization of attention weights is over different time steps based on different heads over all the facts in the story and a question.
Different color bars on the left side indicate attention weights based on different heads (4 heads in total).
An example from tasks 1: (requiring one supportive fact to solve)
Story:
John travelled to the hallway.
Mary journeyed to the bathroom.
Daniel went back to the bathroom.
John moved to the bedroom
Query:
Where is Mary?
Model’s output:
bathroom
(a) Step 1
(b) Step 2
(c) Step 3
(d) Step 4
716:YAMAGUTIseisei
19/08/11 18:08:17.27 TqTByUJCx
Figure 5:
Visualization of the attention distributions, when encoding the question: “Where is Mary?”.
An example from tasks 2: (requiring two supportive facts to solve)
Story:
Sandra journeyed to the hallway.
Mary went to the bathroom.
Mary took the apple there.
Mary dropped the apple.
Query:
Where is the apple?
Model’s output:
bathroom
(a) Step 1 (b) Step 2 (c) Step 3 (d) Step 4
Figure 6:
Visualization of the attention distributions, when encoding the question: “Where is the apple?”.
An example from tasks 2: (requiring two supportive facts to solve)
Story:
John went to the hallway.
John went back to the bathroom.
John grabbed the milk there.
Sandra went back to the office.
Sandra journeyed to the kitchen.
Sandra got the apple there.
Sandra dropped the apple there.
John dropped the milk.
Query:
Where is the milk?
Model’s output:
bathroom
(a) Step 1 (b) Step 2 (c) Step 3 (d) Step 4
717:YAMAGUTIseisei
19/08/11 18:08:49.22 TqTByUJCx
Figure 7:
Visualization of the attention distributions, when encoding the question: “Where is the milk?”.
An example from tasks 3: (requiring three supportive facts to solve)
Story:
Mary got the milk.
John moved to the bedroom.
Daniel journeyed to the office.
John grabbed the apple there.
John got the football.
John journeyed to the garden.
Mary left the milk.
John left the football.
Daniel moved to the garden.
Daniel grabbed the football.
Mary moved to the hallway.
Mary went to the kitchen.
John put down the apple there.
John picked up the apple.
Sandra moved to the hallway.
Daniel left the football there.
Daniel took the football.
John travelled to the kitchen.
Daniel dropped the football.
John dropped the apple.
John grabbed the apple.
John went to the office.
Sandra went back to the bedroom.
Sandra took the milk.
John journeyed to the bathroom.
John travelled to the office.
Sandra left the milk.
Mary went to the bedroom.
718:YAMAGUTIseisei
19/08/11 18:09:46.92 TqTByUJCx
Mary moved to the office.
John travelled to the hallway.
Sandra moved to the garden.
Mary moved to the kitchen.
Daniel took the football.
Mary journeyed to the bedroom.
Mary grabbed the milk there.
Mary discarded the milk.
John went to the garden.
John discarded the apple there.
Query:
Where was the apple before the bathroom?
Model’s output:
office
(a) Step 1
(b) Step 2
(c) Step 3
(d) Step 4
Figure 8:
Visualization of the attention distributions, when encoding the question: “Where was the apple before the bathroom?”.
Generated by LaTeXML [LOGO]
Want to hear about new tools we're making? Sign up to our mailing list for occasional updates.
Subscribe
Built by Andreas Jansson and Ben Firshman, with help from LaTeXML. Contribute on GitHub. Kindly sponsored by YLD.
719:YAMAGUTIseisei
19/08/13 23:49:09.65 RM77FnSdS
>>685
A key consideration in learning algorithms is the issue of generalization, or the ability to robustly deal with novel patterns.
720:YAMAGUTIseisei
19/08/14 12:18:05.06 SHJkZmNqv
>>686
Each distal segment contains a number of synapses, representing lateral connections from a subset of the other NM - 1 cells.
721:YAMAGUTIseisei
19/12/01 01:14:16.63 jGtwQP38C
URLリンク(www.expedient.com)
What Are the Differences Between Backups and Disaster Recovery?
Author:
Erin Masterson
Category:
Disaster Recovery, Data Centers, Infrastructure Availability
Disaster recovery planning is an integral part of any business’s IT strategy, and is becoming more prevalent as security breaches and network outages have become common threats, and the cost of downtime has steadily increased.
In the beginning stages of disaster recovery planning, decision makers are often mistaken about what constitutes a disaster recovery plan.
Many times they are misled by the idea that data backup is sufficient precaution in the event of a disaster.
“Customers often come to us seeking disaster recovery services without realizing that simply backing up their data is not enough,” says Joe Palian, Regional Account Executive at Expedient.
While having a backup strategy is important, it is not the same as a disaster recovery strategy; rather, the beginning stages of establishing a proper DR plan.
A backup is a copy of your data; a disaster recovery plan is insurance that guarantees its recovery.
So, what makes backups and disaster recovery different?
722:YAMAGUTIseisei
19/12/01 01:14:54.79 jGtwQP38C
1.)
Data retention requirements
Backups are typically performed on a daily basis to ensure necessary data retention at a single location, for the single purpose of copying data.
Disaster recovery requires the determination of the RTO (recovery time objective) in order to designate the maximum amount of time the business can be without IT systems post-disaster.
Traditionally, the ability to meet a given RTO requires at least one duplicate of the IT infrastructure in a secondary location to allow for replication between the production and DR site.
2.)
Recovery ability
Disaster recovery is the process of failing over your primary environment to an alternate environment that is capable of sustaining your business continuity.
Backups are useful for immediate access in the event of the need to restore a document, but does not facilitate the failover of your total environment should your infrastructure become compromised.
They also do not include the physical resources required to bring them online.
723:YAMAGUTIseisei
19/12/01 01:15:31.35 jGtwQP38C
3.)
Additional resource needs
A backup is simply a copy of data intended to be restored to the original source.
DR requires a separate production environment where the data can live.
All aspects of the current environment should be considered, including physical resources, software, connectivity and security.
4.)
Planning process
Planning a backup routine is relatively simple, since typically the only goals are to meet the RPO (recovery point objective) and data retention requirements.
A complete disaster recovery strategy requires additional planning, including determining which systems are considered mission critical, creating a recovery order and communication process, and most importantly, a way to perform a valid test.
The overall benefits and importance of a DR plan are to mitigate risk and downtime, maintain compliance and avoid outages.
Backups serve a simpler purpose.
Make sure you know which solution makes sense for your business needs.
Looking to improve your DR preparedness? Follow these 6 Steps.
Have any questions for Erin Masterson?
724:YAMAGUTIseisei
19/12/01 01:16:00.97 jGtwQP38C
the cost of downtime
URLリンク(thecloudcalculator.com)
often mistaken about what constitutes a disaster recovery plan
URLリンク(www.expedient.com)
a disaster recovery strategy
URLリンク(www.expedient.com)
Disaster recovery
URLリンク(www.expedient.com)
the failover of your total environment
URLリンク(www.expedient.com)
a way to perform a valid test
URLリンク(www.expedient.com)
Follow these 6 Steps
URLリンク(bit.ly)
IT Disaster Recovery Planning: What Hurricane Season Can Teach Los Angeles Companies
IT Disaster Recovery Planning:
What Hurricane Season Can Teach Los Angeles Companies
URLリンク(sugarshot.io)
725:YAMAGUTIseisei
19/12/08 21:27:02.54 a72oztLlg
>>721
Customers often come to us seeking disaster recovery services without realizing that simply backing up their data is not enough.
>>724
URLリンク(j.mp) URLリンク(go.expedient.com)
726:YAMAGUTIseisei
20/03/18 17:32:46.47 jXSDxkt5c
URLリンク(pnas.org) pnas.org/content/117/4/1853
A scalable pipeline for designing reconfigurable organisms
View ORCID ProfileSam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard
PNAS first published January 13, 2020 URLリンク(doi.org)
Sam Kriegman
aDepartment of Computer Science, University of Vermont, Burlington, VT 05405;
Douglas Blackiston
bDepartment of Biology, Tufts University, Medford, MA 02153;cAllen Discovery Center, Tufts University, Medford, MA 02153;
Michael Levin
bDepartment of Biology, Tufts University, Medford, MA 02153;cAllen Discovery Center, Tufts University, Medford, MA 02153;dWyss Institute for Biologically Inspired Engineering, Harvard University, Boston, MA 02115
Josh Bongard
aDepartment of Computer Science, University of Vermont, Burlington, VT 05405;
Phase offsets stored in the genotype were mutated by adding a number that was drawn randomly from a normal distribution with mean zero and SD s = 0.4π.
Also, contractile tissue incurs a much higher metabolic cost compared to nonmuscle tissue (the human heart consumes 1 mM ATP per second; ref. 31 ).
Also, contractile tissue incurs a much higher metabolic cost compared to nonmuscle tissue (the human heart consumes ∝1 mM ATP per second; ref. 31 ).
727:YAMAGUTIseisei
20/04/12 14:34:40.36 juW0pBg5d
Memory & Cognition
1974, Vol. 2, No. 3, 467-471
The influence of one memory retrieval on a subsequent memory retrieval *
GEOFFREY R. LOFTUS and ELIZABETH F. LOFTUS
University of Washington, Seattle, Washington 98195
*Requests for reprints may be sent to either Loftus, Department of Psychology, University of Washington, Seattle, Washington 98195.
The research was supported by a National Institute of Mental Health grant to E. Loftus and a National Science Foundation grant to G. Loftus.
Appreciation is expressed to Thomas 0. Nelson for his comments on the manuscript.
Ss produced an instance of a category and following zero or two intervening items produced a second instance of the same category.
The second instance was produced more quickly than the initial instance.
This finding, in conjunction with other data reported in the paper, indicate that the reduction in latency for the second instance is due mostly to a reduction in the rate with which the category is searched.
728:YAMAGUTIseisei
20/04/12 14:35:58.23 juW0pBg5d
In an experiment by Freedman and Loftus (1971), Ss were shown a noun category plus a restricting letter or adjective and were asked to name an instance of the category which began with the letter or which was characterized by the adjective.
Reaction time to produce the response was measured.
The data were discussed in terms of a model that postulated a hierarchical memory composed of noun categories (e.g., animals) with subsets (eg., birds, dogs) and supersets (e.g., living things) of each category.
Retrieval from this hierarchical structure was assumed to consist of at least two major steps: (1) entering the appropriate category and (2) searching the category for an appropriate member.
The times to execute Step 1 and Step 2 are hereafter denoted t1 and t2, respectively.
The duration of t1 was estimated to be about .25 sec by the following reasoning.
Ss saw stimuli presented with the category either first (e.g., fruit-P) or second (e.g., P-fruit) and with at least a 1/2-sec interval between the noun and restrictor.
Reaction times were measured from the presentation of the second member of the pair.
When the category came second, the total retrieval process began only after its presentation and included both t1 and t2, according to the model.
When the category came first, however, t1 could be completed before the restrictor was shown.
For example, given the stimulus fruit-P, the S could enter the category “fruits” during the interval.
Since measured reaction time begins when “P” is presented, measured reaction time excludes t1 in this case.
The decrease in reaction time when the category is shown first vs second can therefore be equated with t1, which is excluded in the former case and included in the latter.
729:YAMAGUTIseisei
20/04/12 14:46:46.31 juW0pBg5d
More recently, Loftus (1973) asked Ss to produce a member of a category and a short time later asked them to produce a different member of that category.
--
This was accomplished by showing a category-letter pair , which asked the S for an appropriate instance, then, following zero, or two intervening items, showing the same category paired with a different letter , which asked for a different instance.
This was accomplished by showing a AB pair (eg., cat-P), which asked the S for an appropriate instance, then, following zero, or two intervening items, showing the same BB paired with a different letter (e.g., cat-A), which asked for a different instance.
--
Interest centered around the question of whether the speed of retrieving the second instance of a category was affected by the retrieval of the first instance and/or the lag between the two retrievals.
The results indicated that response latency for the second instance was shorter than response latency for the first instance and increased monotonically with the number of intervening items.
For example, a S’s baseline time to name a fruit beginning with the letter “P” was 1.52 sec.
However, it took him 1.22 sec to produce the same response if he had named a different fruit on the previous trial and 1.29 sec to produce the response if he had named a different fruit two trials back.
The results of the Loftus (1973) study thus indicate that the process of retrieving information from a category facilitates a subsequent retrieval from that category.
However, in this experiment the S was presented with the category name and restricting letter simultaneously; retrieval time thus included both t1 and t2.
Consequently, the facilitation effect could have involved a reduction in t1 or t2 or both.
The present experiment is designed to distinguish among these three possibilities.
730:YAMAGUTIseisei
20/04/12 15:03:05.89 juW0pBg5d
--
This was accomplished by showing a category-letter pair (eg., fruit-P), which asked the S for an appropriate instance, then, following zero, one, or two intervening items, showing the same category paired with a different letter (e.g., fruit-A),
which asked for a different instance.
--
In some conditions of the present experiment, an interval was inserted between the category name and the letter and the stimuli were presented either in the order category-letter or in the order letter-category
[as in the Freedman & Loftus (1971) study].
As noted above, this procedure allows an estimation of t1 .
Additionally in the present experiment, the S was required to name an instance of a category and shortly thereafter was asked to name a second instance of the category [as in the Loftus (1973) study].
This design is sufficient to determine the locus of the reduction in reaction time to name a second category instance.
Figure 1 shows three possible patterns of results.
Suppose first that only category entry time, t1, is reduced when a second category instance is produced.
In this case, the results shown in Fig.1a should obtain: the letter-category conditions (which include t1) should depend on the prior retrieval, whereas the category-letter conditions (which exclude t1) should not.
467
--
In some conditions of the present experiment, an interval was inserted between the category name and the letter and the stimuli were presented either in the order A in the order B [as in the Freedman & Loftus (1971) study].
731:YAMAGUTIseisei
20/04/12 15:04:33.20 juW0pBg5d
468
LOFTUS AND LOFTUS
a
RT
* letter - category
* category - letter
0 2 initial
LAG
b
RT
0 2 initial
LAG
c
RT
0 2 initial
LAG
Fig.1.
Three possible patterns of results for the relationship between time and the number of intervening items (lag) between two appearances of a critical category.
732:YAMAGUTIseisei
20/04/12 15:05:17.20 juW0pBg5d
Conversely, suppose that only category search time, t2, is reduced when the second category instance is produced.
Such a situation would lead to the results shown in Fig.1b.
Both the category-letter and the letter-category conditions include t2, so they should be affected equally by the initial retrieval.
The final possibility is that both t1 and t2 are reduced.
This situation would predict the results shown in Fig.1c.
Here, the category-letter condition (which includes t2 but not t1) should be affected by the initial retrieval, but the letter-category condition (which includes both t, and t2) should be affected to a greater degree.
METHOD
Subjects
Eighteen Ss from the New School for Social Research received $5 for their participation in two 1-h sessions, which occurred on 2 consecutive days.
No S had previously participated in a memory experiment.
Materials
Each stimulus was printed in block letters on a 5 x 8 in. index card.
A stimulus always consisted of a category name plus a letter (e.g., fruit-P).
Eighty critical category names were selected from the Battig and Montague (1969) and Shapiro and Palermo (1970) category norms.
Each of the category names was paired with two different letters.
If “dominance" is defined as the frequency with which a word is given as an exemplar of a category, then one of the twu category-letter stimuli will be referred to as more dominant than the other.
In addition to the 160 critical stimuli (80 categories each paired with two letters), 80 filler stimuli were used.
The filler stimuli also consisted of a category plus a letter.
Some of the filler categories were used only once; others appeared twice with two different letters.
Thus, each S saw 240 unique stimuli (80 critical categories, each paired with two letters, plus 80 filler stimuli).
733:YAMAGUTIseisei
20/04/12 15:05:56.33 juW0pBg5d
Design
There were three within-S factors: order (category-letter vs letter-category), interval (simultaneous presentation of the stimuli vs 2.5-sec interval between the category name and the letter), and lag (Lag 0, Lag 2, and initial presentation).
These factors were combined factorially, thereby giving a 2 (orders) by 2 (intervals) by 3 (lags) by 18 (Ss) design.
Each S received a different permutation of the 240 items with the following restrictions:
(1)The initial presentation of a critical category-letter pair was followed after zero or two intervening filler items (i.e., at Lag 0 or at Lag 2) by the presentation of the same category paired with a different letter.
Each S received 40 stimuli presented at Lag 0 and 40 at Lag 2.
(2) On half of the trials, Ss saw the stimulus corresponding to the high dominant instance before seeing the stimulus corresponding to the low dominant instance.
For the remaining trials, the reverse arrangement held.
A given category was presented in the order dominant-nondominant for half the Ss and in the reverse order for the remaining half of the Ss.
734:YAMAGUTIseisei
20/04/12 15:06:41.90 juW0pBg5d
Procedure
Each S was told that he would see items consisting of categories and letters and that he was to respond with a word in the category that began with the given letter.
He was given examples and told to respond as quickly as possible, but to avoid errors.
The S sat in front of a screen with a window covered by half-silvered glass.
An index card containing the stimulus was placed in a dark enclosure behind the minor and was presented by illuminating the enclosure.
A microphone was placed in front of the S, and he responded by speaking into it.
A trial consisted of the following:
(a) a card with the item printed in large type was placed in the darkened enclosure;
(b) the E said “ready” and pressed a button which illuminated the first member of the stimulus pair;
(c) either simultaneously or after a 2.5-sec interval, the second member of the pair was automatically illuminated and an electric timer started;
(d) the S’s verbal response activated a voice key that stopped the timer and temrinated the trial.
A warm-up period of 20 trials preceded the experimental trials each day.
735:YAMAGUTIseisei
20/04/12 15:07:21.47 juW0pBg5d
RESULTS
Only correct responses (96%) to the critical stimuli were included in the following analyses.
Median latencies were obtained for each S’s responses in each of the 12 conditions.
For each condition, mean latencies were then obtained by averaging the medians from individual Ss; these means are plotted in Figs. 2 and 3.
Figure 2 shows the results when the 2.5-sec interval was inserted between the category and the letter.
In both the letter-category and category-letter conditions, a second instance of a category is produced faster than the first instance; furthermore, a second instance is produced faster at Lag 0 than at Lag 2.
Figure 3 indicates that the same pattern of results obtains when letter and noun are presented simultaneously.
A 2 (orders) by 2 (intervals) by 3 (lags) analysis of variance was done on the latency data.
Significant effects were found for lag [F(2,34) = 6.57, p < .05], category-letter order [F(1,17) = 14.71, p< .01],‘and interval [F(1,17) = 33.52, p <01].
THE INFLUENCE OF ONE MEMORY RETRIEVAL
469
None of the two-way or three-way interactions was significant (F < 1 for all cases).
736:YAMAGUTIseisei
20/04/12 15:11:11.69 juW0pBg5d
DISCUSSION
Dependence of Memory Retrievals
A number of studies have indicated that the time to retrieve information from a semantic category is decreased if that category has been accessed a short time previously.
Collins and Quillian (1970), for example, have shown that the time required to answer such questions as “Is a canary a bird?” is decreased by as much as 600 msec if information about canaries has been accessed on the previous trial.
Using a somewhat different paradigm, Meyer and Schvaneveldt (Meyer & Schvaneveldt, 1971; Meyer, Schvaneveldt, & Ruddy, 1972', Schvaneveldt & Meyer, 1973; Meyer, 1973) have shown the same thing.
In these experiments, Ss were required to classify letter strings as words or nonwords.
The general finding was that the reaction time to classify a letter string as a word is faster if the S has just classified 3 semantically similar word as opposed to a semantically dissimilar word.
Thus, for example, the time it takes to classify “butter” as a word is faster if “butter” is preceded by “bread” than if it is preceded by “nurse.”
Two general classes of models have been proposed to handle such results.
A location shifting model (Meyer & Schvaneveldt, 1971) assumes that when a S has finished processing a member of a particular category
An activation model, on the other hand, assumes that when items in a category are processed, other items are “excited” or “activated” to the extent that they are semantically similar to the information being processed.
Two further assumptions are made: first (Warren, 1970) that activation decays away over time and second that activated items are more readily accessible than nonactivated items.
--
A location shifting model (1971) assumes that when a S has finished processing a member of a particular category and must then shift to begin processing a second category, the shift time is dependent upon the semantic distance between the two categories.
737:YAMAGUTIseisei
20/04/12 15:13:05.83 juW0pBg5d
2.5 sec. inlerval
RT
1.90
: * Letter - Category
: * Category - Letter
1.60
1.50
0 2 Initial
LAG
Fig. 2.
Mean reaction time in seconds as a function of the number of intervening items (lag) between two appearances of a critical category. Items were presented with a 2.5-sec interval between the category and the letter.
simult.
RT
2.20
: * Letter - Category
: * Category - Letter
1.90
1.80
0 2 Initial
LAG
Fig. 3.
Mean reaction time in seconds as a function of the number of intervening items (lag) between two appearances of a critical category. The category and letter were presented simultaneously.
738:YAMAGUTIseisei
20/04/12 15:13:51.71 juW0pBg5d
The results of the present experiment together with the data of Meyer et al (1972) and Loftus (1973) disconfirm the location shifting model and support the activation model.
All of these experiments involve the following sorts of comparisons.
Let T represent target information whose time to be processed is the dependent variable of interest.
Let R represent information which is semantically related to T, and finally let U1 and U2 represent information which is semantically unrelated to T.
Now consider three conditions:
Condition a: Process U1 ; Process U1 ; Process T.
Condition b: Process R ; Process U2 ; Process T.
Condition c: Process U1 ; Process R ; Process T.
The data show that T is processed fastest in Condition c, next fastest in Condition b, and slowest in Condition a.
Both the location shifting model and the activation model correctly predict that reaction time in Condition c would be faster than reaction time in Conditions a and b.
However, the predictions of the two models differ with regard to the relationship between Conditions a and b.
A location shifting model incorrectly predicts that reaction time would be the same for Conditions a and b, since in both cases the S is shifting from the unrelated category, U2 to T.
An activation model, on the other hand, correctly predicts the obtained pattern of results.
This is because in Condition b, T is assumed to have been activated by R, and this activation has not decayed by the time T is processed.
In Condition a, on the Other hand, T is not assumed to have been activated at all; therefore, time to process T would be longer.
Processing Stages
In the outset of this report, it was noted that the semantic retrieval model proposed by Freedman and Loftus (1971) postulates two major processing stages: entering a category (which takes time t1) and searching the category (which takes time t2).
739:YAMAGUTIseisei
20/04/12 15:14:42.33 juW0pBg5d
470
LOFTUS AND LOFTUS
Table 1
Time Estimates (in Seconds) for Memory Retrieval Stages as a Function of Three Lag Conditions
Retrieval Stage
Lag Condition
Lag 0 Lag 2 Initial
t1
Category entry time
0.20 0.22 0.27
t2 + k
Category search time plus baseline
1.47 1.65 1.69
t3
Eye movement time
0.14 0.14 0.13
t4
Extra encoding time
0.21 0.16 0.22
740:YAMAGUTIseisei
20/04/12 15:15:16.58 juW0pBg5d
Another stage, taking time k, is a baseline stage, involving response execution, etc.
Unfortunately, these stages are not sufficient to handle the data from the present experiment.
To see why this is so, consider the reaction times to initially access a category.
These reaction times fall into a 2 by 2 design with order (category-letter vs letter-category) and interval (2.5 sec vs simultaneous) as factors.
According to the Freedman-Loftus model, the processing times involved in initial access should be as follows:
Condition 1, category-letter; interval: RT1= t2 + k
Condition 2, letter-category; interval: RT2 = t1 + t2 + k
Condition 3, category-letter; simultaneous: RT3 = t1 + t2 + k
Condition 4, letter-category; simultaneous: RT4 = t1 + t2 + k
Thus reaction times for Conditions 2-4 should be equal to each other and should differ (by t1) from the reaction time to Condition 1.
However, the data indicate that all four reaction times differ from one another, thereby necessitating the postulation of additional processing stages.
First, in Condition 4, the predisposition to encode the category before the letter may conflict with normal left-to-right reading habits.
Thus, an additional eye fixation could sometimes occur in Condition4 relative to the other three conditions.
We shall label the time for this additional eye fixation t3.
Secondly, when category and letter are presented simultaneously (Conditions 3 and 4), reaction time must include the time to encode both stimuli.
With a 2.5-sec interval, on the other hand (Conditions 1 and 2), reaction time includes the time to encode only one of the two stimuli.
Let the extra encoding time required in Conditions 3 and 4 be designated by t4.
We are now in a position to include the two new stages in the four initial reaction times.
741:YAMAGUTIseisei
20/04/12 15:18:31.33 juW0pBg5d
(1a) Category-letter; interval:
RT 1: t2 +k=1.69 sec
(1b) Letter-category; interval: RT2=t1+t2 +k=1.9ésec
(1c) Category-letter; simultaneous: RT3=t1 +152 +t4+k=2.18 sec
(1d) Letter-category; simultaneous: RT4 =t1 +132 +t3 +t4 +k=2.31 sec
By appropriate manipulations of Eqs. 1a-4a, we find that t1 = 0.27 sec (RT2 - RTI); (t2 + k) = 1.69 sec (RT1), t3 = 0.13 sec (RT4 - RT3); and t4 = 0.22 sec (RT3 - RT2 ).
The estimate of 0.27 sec for t1 (category entry time) coincides well with previous estimates obtained by Freedman and Loftus (1971) and Loftus and Freedman (1972).
The estimate of 0.22 sec for t4 (encoding time) is far greater than one would expect if “encoding” meant only the process of pattern-recognizing the visual stimulus (cf. Sperling, 1963, who estimated 10msec per item for the pattern-recognition process).
Thus the obtained estimate of 0.22 sec must include a great deal more processing, although it is impossible in the present experiment to determine what such encoding might consist of.
Finally, since an eye fixation usually lasts on the order of 200-300 msec, the estimate of 0.13 sec for t3 (extra fixation time) is somewhat less than one would expect.
A possible reason for this discrepancy is that additional eye fixations may not be made on all of the Condition 4 trials.
The notion of an extra eye fixation sometimes occurring in Condition 4 is, of course, easily testable.
One more parenthetical remark should be made.
As noted above, the interaction of interval time and category-letter order was not significant.
If the null hypothesis of no interaction is accepted, then inspection of Eqs. 1a-4a indicates that t1 = t3.
(This can be seen either by the fact that RT3 - RT1 = RT4 - RT2 or by the fact that RT2 - RT1 = RT4 - RT3, both of which are true under the null hypothesis.)
However, since nothing in the present experiment necessarily warrants acceptance of the null hypothesis, the equality of t1 and t3 should not be taken very seriously.
742:YAMAGUTIseisei
20/04/12 15:22:46.15 juW0pBg5d
What Stage Does Activation Affect?
Using the logic outlined above, it is possible to obtain estimates of t1, (t2 + k), t3, and t4 for second category presentations at Lags 0 and 2.
These estimates, along with the estimates given above for initial presentation, are shown in Table 1.
The statistical analyses of the data indicate that the only parameter which reliably changes over lag condition is t2 + k.
If we make the reasonable assumption that k remains constant over lag conditions, then t2, the category search time, constitutes the locus of the activation effect.
This finding agrees with the conclusion of Meyer (1973, p.30), who noted that “The semantic distance between categories...may affect the search rate for the second category.”
The invariance of encoding time (t4) over lag condition is somewhat at odds with the finding of Meyer et al (1972, Experiment 3) that encoding time appears to be shortened by prior processing of semantically similar information.
The reason for this discrepancy is not entirely clear.
A possible explanation may lie in the fact that the processing delay between the two categories was much shorter in the Meyer et al experiment than in the present experiment,
and the activation decay function for encoding time may be different from the analogous decay function for search rate.
THE INFLUENCE OF ONE MEMORY RETRIEVAL
47l
--
A possible explanation may lie in the fact that the processing delay between the two categories was shorter than the present experiment,and the activation decay function for encoding time may be different from the analogous decay function for search rate
743:YAMAGUTIseisei
20/04/12 15:24:16.79 juW0pBg5d
REFERENCES
Battig, W. F., & Montague, W. E.
Category norms for verbal items in 56 categories: A replication and extension of the Connecticut category norms.
Journal of Experimental Psychology Monograph,
1969. 80(3, Pt.2).
Collins, A. M., & Quillian, M. R.
Facilitating retrieval from semantic memory: The effect of repeating part of an inference.
In A. F. Sanders (Ed.), Attention and performance III.
Amsterdam: North-Holland, 1970.
Freedman, J. L., & Loftus, E. F.
Retrieval of words from long-term memory.
Journal of Verbal Learning & Verbal Behavior,
1971, 10, 107-115.
Loftus, E. F.
Activation of semantic memory.
American Journal of Psychology.
1974.
in press.
Loftus, E. F., & Freedman, J. L.
Effect of category-name frequency on the speed of naming an instance of the category.
Journal of Verbal Learning & Verbal Behavior,
1972, 11, 343-347.
Meyer, D. E.
Correlated operations in searching stored semantic categories.
Journal of Experimental Psychology,
1973, 99, 124-133.
744:YAMAGUTIseisei
20/04/12 15:24:44.18 juW0pBg5d
Meyer, D. E., & Schvaneveldt, R. W.
Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations.
Journal of Experimental Psychology,
1971, 90, 227-234.
Meyer, D. E., Schvaneveldt, R. W., & Ruddy, M. G.
Activation of lexical memory.
Paper presented at the meeting of the Psychonomic Society.
St. Louis, November 1972.
Schvaneveldt, R. W., & Meyer, D. E.
Retrieval and comparison processes in semantic memory.
In S. Kornblum (Ed.), Attention and performance IV.
New York: Academic Press, 1973
Shapiro, S. I., & Palermo, D. S.
Conceptual organization and class membership: Normative data for representatives of 100 categories.
Psychonomic Monograph Supplements, 1970, 3(11, Whole No. 43).
Sperling, G.
A model for visual memory tasks.
Human Factors, 1963, 5, 19-31.
Warren, R. E.
Stimulus encoding and memory.
Unpublished doctoral dissertation, University of Oregon, 1970.
(Received for publication September 17, 1973; revision accepted December 6, 1973.)
745:YAMAGUTIseisei
20/06/19 20:32:52.46 ZY15djw41
>>727-744
URLリンク(link.springer.com)
URLリンク(link.springer.com)
>>732
If “dominance" is defined as the frequency with which a word is given as an exemplar of a category, then one of the two category-letter stimuli will be referred to as more dominant than the other.
>>741
By appropriate manipulations of Eqs 1a-4a, we find that t1 = 0.27 sec (RT2 - RTI); (t2 + k) = 1.69 sec (RT1), t3 = 0.13 sec (RT4 - RT3); and t4 = 0.22 sec (RT3 - RT2 ).
If the null hypothesis of no interaction is accepted, then inspection of Eqs 1a-4a indicates that t, = t3.
746:YAMAGUTIseisei
20/08/28 00:26:14.31 STE/0glun
URLリンク(nature.com)
Thank you for visiting nature.com. You are using a browser version with limited support for CSS.
To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer).
In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
:
* Article
* Open Access
* Published: 25 February 2020
Memristive synapses connect brain and silicon spiking neurons
* Alexantrou Serb1,
* Andrea Corna2,
* Richard George3,
* Ali Khiat1,
* Federico Rocchi2,
* Marco Reato2,
* Marta Maschietto2,
* Christian Mayr3,
* Giacomo Indiveri ORCID: orcid.org/0000-0002-7109-16894,
* Stefano Vassanelli ORCID: orcid.org/0000-0003-0389-80232 &
* Themistoklis Prodromakis ORCID: orcid.org/0000-0002-6267-69091
Scientific Reports volume 10, Article number: 2590 (2020) Cite this article
:
Subjects
* Bionanoelectronics
* Nanosensors
747:YAMAGUTIseisei
20/08/28 00:34:57.19 STE/0glun
Memristors
The memristive synapse set-up consisted of an array of memristive devices positioned inside an ArC memristor characterisation and testing instrument33 (Supplementary Fig.5. http:www.arc-instruments.co.uk).
The instrument is controlled by a PC, which handles all the communications over UDP; all through a python-based user interface.
The software is configured to react to UDP packets carrying information about the firing of either artificial or biological neurons (who fired when).
Once a packet is received,
the ID of the neuron that emitted it and the time of spiking are both retrieved from the packet payload and the neural connectivity matrix is consulted in order to determine which neurons are pre- and which are post-synaptic to the firing cell.
Then, if the plasticity conditions are met, the ArC instrument applies programming pulses that cause the memristive synapses to change their resistive states.
Importantly, the set-up can control whether LTP- or LTD-type plasticity is to be applied in each case, but once the pulses have been applied it is the device responses that determine the magnitude of the plasticity.
Notably, resistivity transitions of the device are non-volatile, they hold over at least hours27 as also exemplified in our prototype experiment and are therefore fully compatible with typical LTP and LTD time scales of natural synapses.
The system is sustained by a specific methodology for handling timing within the overall network (Zurich, Southampton, Padova).
The set-up in Southampton being the node that links Zurich and Padova together, controls the overall handling of time.
--
Once a packet is received, the ID of the neuron that emitted it and the time of spiking are retrieved from the neural connectivity matrix (held at the Southampton set-up) is consulted
the ID of the neuron that emitted it and the time of spiking are both retrieved from the packet payload and the neural connectivity matrix (held at the Southampton set-up) is consulted
748:YAMAGUTIseisei
20/08/28 00:58:25.57 STE/0glun
Under this system, one of the partners (in our case Zurich) is labelled as the “primary partner” and all timing information arriving from that partner is treated as a ground truth.
Every timing information sent by other partners then has to be related to this ground truth, for example if the primary partner says that neuron 12 fires a spike at time 305, then the secondary partner(s) is informed of this (through Southampton).
If then a neuron in the secondary partner set-up fires 5 time units (as measured by a wall-clock) after being informed of the firing of neuron 12, it emits a packet informing Southampton that e.g. neuron 55 fired at time 310.
This way the relative timing between spikes arriving from the primary partner and the spikes triggered by the secondary partner(s) in response is maintained despite any network delays.
The price is that if the secondary partners wish to communicate spikes to the primary partner, network delays for the entire round-trip are then burdening the secondary-to-primary pathway.
The details of timing control at each partner site are fairly complicated and constrained by the set-ups at each partner, but all timing information is eventually encoded in an “absolute time” record held at Southampton.
The rationale behind this design decision was to ensure that at least in the pathway from primary to secondary partner(s) timing control is sufficiently tight to sustain plasticity in the face of network delays.
Neuronal culture and electrophysiology
Embryonic (E18) rat hippocampal neurons were plated and cultured on the CMEA according to procedures described in detail in34.
Recordings were performed on 812 DIV neurons.
The experimental setup in UNIPD(Supplementary Fig.1)enabled UDP-triggered capacitive stimulation of neurons13 while simultaneously recording and communicating via UDP the occurrence of depolarisations that were measured by patch-clamp whole-cell recording
749:YAMAGUTIseisei
20/08/28 01:40:48.13 STE/0glun
The CMEA (20 × 20 independent TiO2 capacitors, each one of area 50 × 50 μm2) was controlled by a dedicated stimulation board and all the connections to partners, Southampton and Zurich, were managed by a PC running a LabVIEW-based software
(National Instruments Corp, Austin, TX, USA).
The stimulation protocol was derived from13 and further optimized for non-invasive adjustable stimulation of the neurons.
In brief, capacitive stimulation was adjusted to the memristor’s resistance (i.e. the synaptor weight) by varying the repetition number of appropriate stimulation waveforms (Supplementary Fig.1).
Patch-Clamp recordings were performed in whole-cell current-clamp configuration using an Axopatch 200B amplifier ( USA) connected to the PC through a BNC-2110 Shielded Connector Block ( TX, USA) along with a PCI-6259 PCI Card ( TX, USA).
WinWCP (Strathclyde Electrophysiology Software, University of Strathclyde, Glasgow, UK) was used for data acquisition.
Micropipettes were pulled from borosilicate glass capillaries (GB150T-10, Science Products GmbH, Hofheim, Germany) using a P-97 Flaming/Brown Micropipette Puller (Sutter Instruments Corp., Novato, CA, USA).
Intracellular pipette solution and extracellular solution used during the experiments were respectively (in mM): 6.0 KCl, 120 K gluconate, 10 HEPES, 3.0 EGTA, 5 MgATP, 20 Sucrose (K); 135.0 NaCl, 5.4 KCl, 1.0 MgCl2, 1.8 CaCl2, 10.0 Glucose, 5.0 HEPES (N).
Digitised recordings were analysed by a custom LabVIEW software running on the PC, allowing detection and discrimination of firing and EPSP activity through a thresholding approach.
All experiments were performed in accordance with the Italian and European legislation for the use of animals for scientific purposes and protocols approved by the ethical committee of the University of Padova and by the Italian Ministry of Health
(authorisation number 522/2018-PR).
--
Molecular Devices, USA
National Instruments Corp, Austin, TX, USA
adjusted to pH 7.3 with 1N KOH
750:YAMAGUTIseisei
20/08/28 01:44:33.73 STE/0glun
References
1.
O’Doherty, J. E. et al.
Active tactile exploration using a brain-machine-brain interface.
Nature 479, 228-231 (2011).
* ADS * Article * Google Scholar
2.
Hampson, R. E. et al.
Developing a hippocampal neural prosthetic to facilitate human memory encoding and recall.
J. Neural Eng. 15, 036014 (2018).
* ADS * Article * Google Scholar
3.
Thakor, N. V.
Translating the Brain-Machine Interface.
Sci. Transl. Med. 5, 210ps17-210ps17 (2013).
* Article * Google Scholar
4.
Mead, C. Neuromorphic electronic systems.
Proc. IEEE 78, 1629-1636 (1990).
* Article * Google Scholar
5.
Vassanelli, S. & Mahmud, M.
Trends and Challenges in Neuroengineering: Toward “Intelligent” Neuroprostheses through Brain-“Brain Inspired Systems” Communication.
Front. Neurosci. 10 (2016).
6.
Boi, F. et al. A Bidirectional Brain-Machine Interface Featuring a Neuromorphic Hardware Decoder.
Front. Neurosci. 10 (2016).
>>748
Recordings were performed on 8-12 DIV neurons.
751:YAMAGUTIseisei
20/08/28 01:45:59.67 STE/0glun
7.
Wei, S. L. et al.
Emulating long-term synaptic dynamics with memristive devices.
ArXiV. 1509, 01998 (2015).
* Google Scholar
8.
Berdan, R. et al.
Emulating short-term synaptic dynamics with memristive devices.
Scientific reports. 6 (2015).
9.
Burr, G. W. et al.
Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element.
IEEE Trans. Electron Devices 62, 34983507 (2015).
* ADS * Article * Google Scholar
10.
Yang, J. J., Strukov, D. B. & Stewart, D. R.
Memristive devices for computing.
Nat. Nanotechnol. 8, 13-24 (2013).
* ADS * CAS * Article * Google Scholar
11.
Gupta, I. et al.
Real-time encoding and compression of neuronal spikes by metal-oxide memristors.
Nat. Commun. 7, 12805 (2016).
* ADS * CAS * Article * Google Scholar
12.
Birmingham, K. et al.
Bioelectronic medicines: a research roadmap.
Nat. Rev. Drug Discov. 13, 399-400 (2014).
* CAS * Article * Google Scholar
>>749
6.0 KCl, 120 K gluconate, 10 HEPES, 3.0 EGTA, 5 MgATP, 20 Sucrose (adjusted to pH 7.3 with 1N KOH); 135.0 NaCl, 5.4 KCl, 1.0 MgCl2, 1.8 CaCl2, 10.0 Glucose, 5.0 HEPES (adjusted to pH 7.4 with 1N NaOH).
752:YAMAGUTIseisei
20/08/28 01:49:08.05 STE/0glun
13.
Schoen, I. & Fromherz, P.
Extracellular Stimulation of Mammalian Neurons Through Repetitive Activation of Na+ Channels by Weak Capacitive Currents on a Silicon Chip.
J. Neurophysiol. 100, 346-357 (2008).
* Article * Google Scholar
14.
George, R., Mayr, C., Indiveri, G. & Vassanelli, S.
Event-based softcore processor in a biohybrid setup applied to structural plasticity.
In 2015 International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP) 1-4, URLリンク(doi.org) (IEEE, 2015).
15.
Rast, A. D. et al.
A location-independent direct link neuromorphic interface.
In The 2013 International Joint Conference on Neural Networks (IJCNN) 1-8, URLリンク(doi.org) (IEEE, 2013).
16.
Keren, H., Partzsch, J., Marom, S. & Mayr, C. G.
A Biohybrid Setup for Coupling Biological and Neuromorphic Neural Networks.
Front. Neurosci. 13 (2019).
17.
Dudek, S. M. & Bear, M. F.
Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade.
Proc. Natl. Acad. Sci. USA 89, 4363-4367 (1992).
* ADS * CAS * Article * Google Scholar
18.
Cooper, L. N. & Bear, M. F.
The BCM theory of synapse modification at 30: interaction of theory with experiment.
Nat. Rev. Neurosci. 13, 798-810 (2012).
* CAS * Article * Google Scholar
753:YAMAGUTIseisei
20/08/28 01:51:23.51 STE/0glun
19.
Vassanelli, S., Mahmud, M., Girardi, S. & Maschietto, M.
On the Way to Large-Scale and High-Resolution Brain-Chip Interfacing.
Cogn. Comput. 4, 71-81 (2012).
* Article * Google Scholar
20.
Giacomello, M. et al.
Stimulation of Ca2+ signals in neurons by electrically coupled electrolyte-oxide-semiconductor capacitors.
J. Neurosci. Methods 198, 1-7 (2011).
* CAS * Article * Google Scholar
21.
Spira, M. E. & Hai, A.
Multi-electrode array technologies for neuroscience and cardiology.
Nat. Nanotechnol. 8, 83 (2013).
* ADS * CAS * Article * Google Scholar
22.
Alivisatos, A. P. et al.
Nanotools for Neuroscience and Brain Activity Mapping.
ACS Nano 7, 1850-1866 (2013).
* CAS * Article * Google Scholar
23.
Angle, M. R., Cui, B. & Melosh, N. A.
Nanotechnology and neurophysiology.
Curr. Opin. Neurobiol. 32, 132-140 (2015).
* CAS * Article * Google Scholar
24.
Duan, X. & Lieber, C. M.
Nanoscience and the nano-bioelectronics frontier.
Nano Res. 8, 1-22 (2015).
* Article * Google Scholar
754:YAMAGUTIseisei
20/08/28 01:52:26.41 STE/0glun
25.
Brivio, S. et al.
Experimental study of gradual/abrupt dynamics of HfO2-based memristive devices.
Appl. Phys. Lett. 109, 133504 (2016).
* ADS * Article * Google Scholar
26.
Serrano-Gotarredona, T., Masquelier, T., Prodromakis, T., Indiveri, G. & Linares-Barranco, B.
STDP and STDP variations with memristors for spiking neuromorphic learning systems.
Front. Neurosci. 7 (2013).
27.
Serb, A. et al.
Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses.
Nat. Commun. 7 (2016).
28.
Qiao, N. et al.
A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses.
Front. Neurosci. 9, 141 (2015).
* Article * Google Scholar
29.
Boegerhausen, M., Suter, P. & Liu, S.-C.
Modeling Short-Term Synaptic Depression in Silicon. Neural Comput.
15, 331-348 (2003).
* Article * Google Scholar
30.
Mitra, S., Fusi, S. & Indiveri, G.
Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI.
IEEE Trans. Biomed. Circuits Syst. 3, 32-42 (2009).
* CAS * Article * Google Scholar
755:YAMAGUTIseisei
20/08/28 01:54:02.52 STE/0glun
31.
Livi, P. & Indiveri, G.
A current-mode conductance-based silicon neuron for address-event neuromorphic systems.
In 2009 IEEE International Symposium on Circuits and Systems 2898-2901 URLリンク(doi.org) (IEEE, 2009).
32.
Deiss, S., Douglas, R. & Whatley, A.
A pulse-coded communications infrastructure for neuromorphic systems.
Pulsed Neural Netw. 157-178 (1999).
33.
Berdan, R. et al.
A u-Controller-Based System for Interfacing Selectorless RRAM Crossbar Arrays.
IEEE Trans. Electron Devices 62, 2190-2196 (2015).
* ADS * CAS * Article * Google Scholar
34.
Antonucci, D. E., Lim, S. T., Vassanelli, S. & Trimmer, J. S.
Dynamic localization and clustering of dendritic Kv2.1 voltage-dependent potassium channels in developing hippocampal neurons.
Neuroscience 108, 69-81 (2001).
* CAS * Article * Google Scholar
35.
Indiveri, G. et al.
Neuromorphic silicon neuron circuits.
Front. Neurosci. 5, 73 (2011).
* PubMed * PubMed Central * Google Scholar
36.
Stathopoulos, S. et al.
Multibit memory operation of metal-oxide bi-layer memristors.
Sci. Rep. 7 (2017).
756:YAMAGUTIseisei
20/08/28 01:54:56.67 STE/0glun
Download references
Author information
Affiliations
1.
Centre for Electronics Frontiers, University of Southampton, Southampton, SO17 1BJ, UK
* Alexantrou Serb
* , Ali Khiat
* & Themistoklis Prodromakis
2.
Biomedical Sciences and Padua Neuroscience Center, University of Padova, Padova, 35131, Italy
* Andrea Corna
* , Federico Rocchi
* , Marco Reato
* , Marta Maschietto
* & Stefano Vassanelli
3.
Institute of Circuits and Systems, TU Dresden, Dresden, 01062, Germany
* Richard George
* & Christian Mayr
4.
Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, 8057, Switzerland
* Giacomo Indiveri
757:YAMAGUTIseisei
20/08/28 01:55:43.29 STE/0glun
Contributions
The experiments were jointly conceived by T.P., S.V. and G.I., who share senior authorship.
The experiments were jointly designed and ran by A.S., A.C., R.G., who are acknowledged as shared first authors.
A.K. manufactured the memristive devices.
FR and MR assisted with the biological system set-up and operation.
MM cultured neurons on chips.
C.M. provided valuable feedback and guidance during the write-up of the paper.
The paper was jointly written by all co-authors.
Corresponding authors
Correspondence to Stefano Vassanelli or Themistoklis Prodromakis.
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
URLリンク(static-content.springer.com)
758:YAMAGUTIseisei
20/08/28 02:10:03.84 STE/0glun
Rights and permissions
Open Access
This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format,as long as you give appropriate credit to the original author(s) and the source,provide a link to the Creative Commons license,and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material.
If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this license, visit URLリンク(creativecommons.org) .
Reprints and Permissions
About this article
Verify currency and authenticity via CrossMark
Cite this article
Serb, A., Corna, A., George, R. et al. Memristive synapses connect brain and silicon spiking neurons. Sci Rep 10, 2590 (2020). URLリンク(doi.org)
Download citation
* Received: 22 October 2019
* Accepted: 21 January 2020
* Published: 25 February 2020
* DOI: URLリンク(doi.org)
Provided by the Springer Nature SharedIt content-sharing initiative
--
This article is licensed under a Creative Commons Attribution 4.0 License, which permits use, sharing and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) , and indicate if changes .
759:YAMAGUTIseisei
21/09/07 11:21:59.91 Sg5KSVwHZ
sage
760:オーバーテクナナシー
21/09/14 07:52:03.64 lSdSBXgiV
UNIVERSAL TRANSFORMERS. Published as a conference paper at ICLR 2019. URLリンク(arxiv-vanity.com) URLリンク(arxiv.org)
Mostafa Dehghani* † Stephan Gouws* Oriol Vinyals
University of Amsterdam DeepMind DeepMind
dehghani@uva.nl sgouws@google.com vinyals@google.com
Jakob Uszkoreit ukasz Kaiser
Google Brain Google Brain
usz@google.com lukaszkaiser@google.com
D.4. LEARNING TO EXECUTE (LTE).
LTE is a set of tasks indicating the ability of a model to learn to execute computer programs and was proposed by Zaremba & Sutskever (2015).
These tasks include two subsets:
1) program evaluation tasks (program, control, and addition) that are designed to assess the ability of models for understanding numerical operations, if-statements, variable assignments, the compositionality of operations, and more, as well as
2) memorization tasks (copy, double, and reverse).
The difficulty of the program evaluation tasks is parameterized by their length and nesting.
The length parameter is the number of digits in the integers that appear in the programs (so the integers are chosen uniformly from [1, length]), and the nesting parameter is the number of times we are allowed to combine the operations with each other.
Higher values of nesting yield programs with deeper parse trees.
For instance, here is a program that is generated with length = 4 and nesting = 3.
Input:
j=8584
for x in range(8):
j+=920
b=(1500+j)
print((b+7567))
Target:
25011
1) program evaluation tasks (A) that are designed to assess the ability of models for understanding numerical operations, if-statements, variable assignments, the compositionality of operations, and more, as well as 2) memorization tasks (B).
761:オーバーテクナナシー
21/09/14 08:02:31.32 lSdSBXgiV
>>760
ukasz Kaiser.
--
Input:
j=8584
for x in range(8):
j+=920
b=(1500+j)
print((b+7567))
Target:
25011
762:YAMAGUTIseisei
22/05/29 03:53:57.30 npUmdHxq/
URLリンク(webcache.googleusercontent.com)
This is the html version of the file URLリンク(www.lst.ethz.ch)
Google automatically generates html versions of documents as we crawl the web.
Page 1
CellVM: A Homogeneous Virtual Machine Runtime System for a Heterogeneous Single-Chip Multiprocessor
Albert Noll ETH Zurich albert.nollaTinf.ethz ch
Andreas Gal University of California, Irvine galATuci edu
Michael Franz University of California, Irvine franzATuci edu
The assign and method benchmark, on the other hand, include CellVM’s worst case scenario: synchronized methods and data structures.
Figure 4.
Performance evaluation of low-level VM operations.
Values are normalized to JamVM running on the PPE