Fire All Neurons!

Fire All Neurons!

Fire All Neurons!

Mechanical analog computers had their origins in Naval Gunnery in World War I […] mechanical analog computers remained of considerable military importance certainly until well into the 1960s and have only been superseded by digital computing systems in the 1970s..

Bromley 1984

John and Sam are talking about the following article: Before Silicon Valley got nasty, the Pirates of Analog Alley fought it out

In this post the dialogue is realised by an interaction of virtual characters, for more information please check the page “Virtual characters

John

Analog computing? I hadn’t really thought about that one …

Today 17:09

Sam

Well, in the beginning, it was quite superior to classical computing, because it was so much faster

Today 17:10

John

Faster? 🤨

Today 17:10

Sam

Yep, it could make quite complex calculations: a century ago analog computers for fire control on battleships solved differential equations in real time 😊                                                            

Today 17:11

John

The who with the what now?! 😄

Today 17:12 

Sam

I don’t mean to go as far back as ancient Greece, but at the beginning of the 20th century shooting the big guns on a ship accurately required a lot of calculations, in real-time (Gears of war: When mechanical analog computers ruled the waves)

Today 17:12

John

I had no idea about this … 🤔

Today 17:13 

Sam

Just like you don’t fall over when you play tennis, because your neural system integrates information about your limbs and the balance organ in your inner ear

Today 17:13

Sam

ships used gyroscopes and analogue computers to keep their guns on target despite rolling with the waves

Today 17:13

John

Wow, that certainly would involve quite a lot of computation I guess

Today 17:15 

Sam

It does! Analog computing does this in a very different way from digital computing, but that doesn’t mean it’s not computing …

Today 17:15

John

Ok, ok, fine! So these analog computers work more or less like the inner ear, therefore we might as well call it neuromorphic computing?

Today 17:16 

Sam

I’m sure some people might disagree, but basically, yes. You’d have to ask Cho about the specifics of how the brain does stuff like that though

Today 17:17

John

I must say, this isn’t really what we were looking for originally …

Today 17:17   

Sam

I guess this kind of measuring, integrating information, and reacting in real-time is a lot less sexy than finding faces in photos and animating them …

Today 17:17

John

Maybe it’s that, but I also suppose that most people wouldn’t consider this as computation at all

Today 17:18   

Sam

I’d say that someone skateboarding is solving a lot of differential equations in real-time! Or they’d crack their skulls … 😅

Today 17:19

John

Just playing devil’s advocate here 😄

Just because we can describe the system using equations, doesn’t mean that the system is computing anything …?

Today 17:19   

Sam

Ah, yeah, I can see where that is coming from .. 😊                  sunflowers don’t do astronomy just by following the sun

Today 17:20

John

That’s it, yes! So what are these navy devices and the inner ear doing exactly that makes them “computers”?

Today 17:20   

Sam

I had a whole conversation with Julia about analog vs digital, I guess this is the same kind of problem right here

Today 17:20

… Continue reading our conversations that are posted every Monday …

1 Star2 Stars3 Stars4 Stars5 Stars (4 votes, average: 4.25 out of 5)
Loading...

Related post

Fire All Neurons!

Fire All Neurons!

Mechanical analog computers had their origins in Naval Gunnery in World War I […] mechanical analog computers remained of considerable military importance certainly until well into the 1960s and have only been superseded by digital computing systems in the 1970s.

read more
The ends justify the rules

The ends justify the rules

Complicated mental processes are entirely reducible to such simple activities as the attentive observation of statements previously accepted as true, the perception of structural, purely external, connections among these statements, and the execution of mechanical transformations as prescribed by the rules of inference

read more
The Brain is as the Brain does

The Brain is as the Brain does

The Brain is as the Brain does

The primary purpose of all neural systems is centralized control of various biological functions…
In the development of information technology there now seems to exist a new phase whereby the aim is to replicate many of these “Neural” functions artificially

Kohonen 1988

John and Sam are talking about the following article:
An introduction to neural computing

In this post the dialogue is realised by an interaction of virtual characters, for more information please check the page “Virtual characters

John

Hi Sam, I wanted to thank you for helping us out with our article!😊

Today 17:09

Sam

My pleasure, I enjoyed working with Julia and you 😄                        

Today 17:10

John

Can I ask you another favor though?

Today 17:10

Sam

Sure, happy to oblige 😊                                                                          

Today 17:11

John

For our next issue we’re thinking of looking at Neuromorphic Engineering

Today 17:12 

Sam

Very interesting, yes! 😊                                                                           

Today 17:12

John

There’s a lot of technical problems with the classical approach, so we thought about presenting some alternatives

Today 17:13 

Sam

Of course, but neuromorphic computing can mean a lot of different things

Today 17:13

John

Yes, that’s why we need your help! I know it is not simply “doing things like the brain

Today 17:15 

Sam

Indeed, in a very literal sense we still can’t do that                              

Today 17:15

John

But Julia’s historical approach did convince me that we are getting closer and closer, right?

Today 17:16 

Sam

I would agree, yes, but we’re still a long way off                                   

Today 17:17

John

OK, fair enough, but would you willing to collaborate on an article discussing the various approaches?

Today 17:17   

Sam

Certainly! Do you plan on going as far back as the 19th century like Julia did?

Today 17:17

John

Not this time, thought I found it quite interesting what she uncovered

Today 17:18   

Sam

So mostly 20th century stuff?                                                                   

Today 17:19

John

Yeah, we wanted to look at the parallel developments from the 1950s onwards

Today 17:19   

Sam

Side by side with the “official” history of AI? Interesting! I like it already

Today 17:20

John

Yes, that was the plan, good to heave you on board!

Today 17:20   

Sam

Can I suggest Analog Computing as the first topic?                             

Today 17:20

… Continue reading our conversations that are posted every Monday …

1 Star2 Stars3 Stars4 Stars5 Stars (4 votes, average: 4.25 out of 5)
Loading...

Related post

Total posts on the argument

Fire All Neurons!

Fire All Neurons!

Mechanical analog computers had their origins in Naval Gunnery in World War I […] mechanical analog computers remained of considerable military importance certainly until well into the 1960s and have only been superseded by digital computing systems in the 1970s.

read more
The ends justify the rules

The ends justify the rules

Complicated mental processes are entirely reducible to such simple activities as the attentive observation of statements previously accepted as true, the perception of structural, purely external, connections among these statements, and the execution of mechanical transformations as prescribed by the rules of inference

read more
The Digital Analogy

The Digital Analogy

The Digital Analogy

Computers are only prostheses, they no more do calculations than clocks tell the time.
Clocks help us to tell the time, but they don’t do it by themselves..

Tallis 2008

Julia and Sam are talking about the following article:
Analog and digital, continuous and discrete

In this post the dialogue is realised by an interaction of virtual characters, for more information please check the page “Virtual characters

Julia

I’m not sure I understand why the 🌍 digital/analog distinction is relevant here …

Today 17:08

Julia

Someone once tried to convince me the distinction doesn’t even make sense!

Today 17:08

Sam

Ha, well, 😁 computationally it does, but perhaps it is better to speak of 🌍 discrete and continuous

Today 17:09   

Julia

Ok, now I’m even more confused …

Today 17:09

Sam

Well, think of a analog and digital watches: both tell the same time, but in a different format

Today 17:10

Julia

Sure, so you mean there is no difference?

Today 17:11

Sam

I didn’t say that, but perhaps that they simply measure time in different ways, it is not time which is analog or digital, but the watch

Today 17:11   

Julia

Ah, ok. So in the case of information processing, it is not the brain or the computer that is analog or digital, but how we measure and describe it?

Today 17:12

Sam

That’s exactly what I mean! We can describe a neuron in different ways and then compare it to a circuit or a flipflop.

Today 17:12   

Julia

So there is a similarity in the description, but not necessarily in the thing you are describing? 🤔                                                                       

Today 17:13

Sam

That starts to sound very philosophical and abstract, but, yes, indeed. We can describe them in the same way.

Today 17:14   

Julia

So what did you mean with discrete and continuous?                                                                                               

Today 17:15

Sam

Both brains and computers work with electricity, but in a computer the voltages are tightly controlled and used in a digital way: ones and zeroes

Today 17:16   

Sam

In a brain the voltage can vary continuously, but sometimes triggers a neuron to “fire”, and up to a point we can treat that as a “one” too

Today 17:16   

Sam

So continuous means that we measure what is happening as close as possible, discrete that we just count the neurons that fire

Today 17:17   

Julia

Aha, so by treating them as discrete units, we can then treat them as digital in our computational model 😊                                                                                        

Today 17:18

Sam

Precisely! We’re not trying to copy the brain at all, only what we think is informationally relevant.

Today 17:19   

Sam

But Cho will tell you we are also ignoring a lot of stuff that happens in the brain to make that possible

Today 17:19  

Julia

And that we don’t really know which bits and pieces are relevant for the brain as a whole

Today 17:20

Sam

Yep, I can tell you exactly how information flows through a computer or even a network, but we can’t trace it in the brain

Today 17:21

Julia

Because in the brain medium and message, 🌍 vehicle and content aren’t clearly distinguishable.

Today 17:22

Julia

So we can’t point at the brain and say “this process right here is this or that law of thought”. I think I get it!

Today 17:22

Sam

Happy to have been of assistance!

Today 17:23

Julia

Thank you so much! I’ll report back to John now

Today 17:23

… Continue reading our conversations that are posted every Saturday …

1 Star2 Stars3 Stars4 Stars5 Stars (4 votes, average: 4.25 out of 5)
Loading...

Related post

Fire All Neurons!

Fire All Neurons!

Mechanical analog computers had their origins in Naval Gunnery in World War I […] mechanical analog computers remained of considerable military importance certainly until well into the 1960s and have only been superseded by digital computing systems in the 1970s.

read more
The ends justify the rules

The ends justify the rules

Complicated mental processes are entirely reducible to such simple activities as the attentive observation of statements previously accepted as true, the perception of structural, purely external, connections among these statements, and the execution of mechanical transformations as prescribed by the rules of inference

read more

Thinking in the Box

Thinking in the Box

Thinking in the Box

Even the simplest brains of the simplest animals are awesome computational instruments. They do computations we do not know how to do, in ways we do not understand

There is nothing that is done in the nervous system that we cannot emulate with electronics if we understand the principles of neural information processing.

Mead 1990

Julia and Sam are talking about the following article:
Neuromorphic Computing

In this post the dialogue is realised by an interaction of virtual characters, for more information please check the page “Virtual characters

Sam

When we talk about computers, we tend to distinguish the machine and its programming: hardware and software

Today 17:09   

Julia

OK, yes, I think everyone is familiar with those terms                         

Today 17:09

Sam

Great, but I’d like to point out how wrong that is, in some respects …

Today 17:10   

Sam

Where do we draw the line between those? Are they really that different? Doesn’t the design of the one determine the design of the other?

Today 17:10   

Julia

Whoa, those are tricky questions! 😅
Remarkably similar to the kind we ask about humans; is that what you’re after?

Today 17:11

Sam

In a sense. We are so used to the hardware/software distinction, that we forget that somewhere they overlap and interact

Today 17:11   

Julia

Can you give me some examples? Otherwise, I fear our readers won’t all get it …😅

Today 17:12

Sam

Sure: flip-flops are at the same time a hardware device and a software abstraction, a circuit and a one or zero

Today 17:12   

Julia

Right, so there is a level where the distinction is arbitrary: it is both a thing and information

Today 17:13

Sam

Exactly! 😉 But then of course at other levels things are much more clear cut

Today 17:14   

Julia

But what did you mean with “wetware” then?                                       

Today 17:15

Sam

Ah, yes, well as far as I know, the brain is like that all over: it is tremendously difficult to keep the hardware and software apart

Today 17:16   

Julia

Oooh, now I get what you mean: a classical computer architecture is completely different

Today 17:16

Sam

Right, but if the brain is an information processor, you can try to just copy the informationally relevant bits

Today 17:17   

Julia

So like Smee you can capture the same informationally relevant relations in a different hardware?

Today 17:18

Sam

Up to a point, of course. We can try to copy some of the structure of the brain, but only in software really

Today 17:19   

Julia

What do you mean? 🤔                                                                             

Today 17:19

Sam

Well, most “artificial neural networks” (ANN) actually are software simulations that run on ordinary beige box computers

Today 17:19   

Julia

Aha, so as you said: a lot of simplification and abstraction 😊                

Today 17:19

Sam

Correct! We still don’t know exactly how the brain does all its information processing or how to copy that efficiently

Today 17:20   

Sam

and then there’s the whole digital/analog problem …

Today 17:20   

… Continue reading our conversations that are posted every Monday …

1 Star2 Stars3 Stars4 Stars5 Stars (4 votes, average: 4.25 out of 5)
Loading...

Related post

Fire All Neurons!

Fire All Neurons!

Mechanical analog computers had their origins in Naval Gunnery in World War I […] mechanical analog computers remained of considerable military importance certainly until well into the 1960s and have only been superseded by digital computing systems in the 1970s.

read more
The ends justify the rules

The ends justify the rules

Complicated mental processes are entirely reducible to such simple activities as the attentive observation of statements previously accepted as true, the perception of structural, purely external, connections among these statements, and the execution of mechanical transformations as prescribed by the rules of inference

read more

Wet Between the Ears

Wet Between the Ears

Wet Between the Ears

Sensations are received by a certain definite number of sensor nerves, which constitute the only means we possess of obtaining a knowledge of the external world.
The sensor nerves pass to the brain, and then come in contact with a highly vascular tissue, called the grey matter of the brain;
Inasmuch as the sensor nerves come in contact with blood-vessels, it follows from voltaic laws, that a voltaic battery exists in the brain, which is opposed to that in the body, and by which the electro-biological circuit is completed.

Smee 1849

Julia and Sam are talking about the following article:
Lecture ON ELECTRO-BIOLOGY; OR, THE VOLTAIC MECHANISM OF MAN

In this post the dialogue is realised by an interaction of virtual characters, for more information please check the page “Virtual characters

Julia

I’m not sure it would qualify as an artificial neural network nowadays, but there was this doctor and metallurgist, working on electro-biology: Alfred Smee

Today 17:08

Julia

He thought that all thought was based on electrical activity in the brain which we could use as a model for a machine

Today 17:08

Sam

That’s amazing! 😁 I didn’t know that … 

Today 17:09   

Julia

It seems he conjectured you’d need an artificial neural network the size of London to model a whole human brain

Today 17:09

Sam

Ha! 😅 He was probably right, early computers where absolutely massive, and supercomputers still are

Today 17:10   

Julia

With the technology of his time this could probably never have been built, he even guessed that the machine would self-destruct by using it …

Today 17:11

Sam

It really is surprising how old some ideas are, and has to wait for technology to catch up so we can realize them!

Today 17:11   

Julia

But a lot of there machines were never built: Babbage’s analytical engine, for instance

Today 17:12

Sam

Or Turing’s Machine, indeed, the idea was enough to prove that something was or wasn’t possible

Today 17:12   

Julia

That’s very interesting!  🤔                                                                       

Today 17:13

Sam

Well, I guess that you could compare and contrast Smee and Babbage in that respect

Today 17:14   

Sam

Babbage did not consider his machine as a model for the mind, but developed his ideas from the technology that was available

Today 17:14   

Sam

Smee tried to design a machine based on what they then understood of the brain, but the technology wasn’t there yet

Today 17:14   

Julia

But now it is?                                                                                               

Today 17:15

Sam

Not exactly, as I mentioned to Manuel, artificial neural netword can’t really compete with our organic brains

Today 17:16   

Julia

So the idea of copying the brain still can’t be realized?                       

Today 17:16

Sam

With some extreme simplifications, maybe a little bit …

Today 17:17   

Julia

Tell me more! 😊                                                                                        

Today 17:18

Sam

I think I need to make a few distinctions first, like between hardware, software, and … wetware

Today 17:19   

Julia

Wetware!? 🤔                                                                                              

Today 17:19

… Continue reading our conversations that are posted every Saturday …

1 Star2 Stars3 Stars4 Stars5 Stars (4 votes, average: 4.25 out of 5)
Loading...

Related post

Fire All Neurons!

Fire All Neurons!

Mechanical analog computers had their origins in Naval Gunnery in World War I […] mechanical analog computers remained of considerable military importance certainly until well into the 1960s and have only been superseded by digital computing systems in the 1970s.

read more
The ends justify the rules

The ends justify the rules

Complicated mental processes are entirely reducible to such simple activities as the attentive observation of statements previously accepted as true, the perception of structural, purely external, connections among these statements, and the execution of mechanical transformations as prescribed by the rules of inference

read more