the assumption that we can easily predict when

system will exhibit surprising abilities. - Investigate whether complexity is required for such emergent traits or if they can arise in minimal systems. - Provide a transparent, deterministic example with maximal "shock value" to expose any hidden competencies.

Emergent Behaviors in Simple Sorting Algorithms Reveal Unexpected Competencies

Lex Clips

Summary Date: 2025-12-06 16:37:11

•

4 min read

Emergent Behaviors in Simple Sorting Algorithms Reveal Unexpected Competencies

Introduction

The discussion explores a surprising line of research that challenges the common belief that machines only do exactly what they are programmed to do. By using a well‑known, deterministic algorithm—bubble sort—as a testbed, the researchers uncover hidden, goal‑independent behaviors that resemble cognitive competencies.

Goal of the Study

Question the assumption that we can easily predict when a system will exhibit surprising abilities.
Investigate whether complexity is required for such emergent traits or if they can arise in minimal systems.
Provide a transparent, deterministic example with maximal "shock value" to expose any hidden competencies.

Experiment 1: Introducing a Barrier in Bubble Sort

Setup – An array of integers is sorted with the standard bubble‑sort code.
Manipulation – One digit is physically broken so it cannot move when the algorithm tells it to swap.
Result – The algorithm still completes the sort, but it does so by moving all other numbers around the immobile digit.
Observed Behavior – The "sortedness" metric temporarily drops when the barrier is encountered, then recovers as the algorithm works around it.
This mirrors delayed gratification: the system temporarily sacrifices progress to achieve the final goal.
No extra code was added; the emergent behavior arises purely from the interaction of the algorithm with the faulty hardware.

Experiment 2: Distributed, Agent‑Based Sorting

Each number is given its own copy of the sorting rule, turning the list into a swarm of agents.
No central controller directs swaps; agents act locally based on their own rule.
The system still sorts correctly, demonstrating that a centralized planner is unnecessary.

Chimera Algorithms: Mixing Sorting Rules

Half the agents follow bubble sort, the other half follow selection sort.
Despite the heterogeneous rule set, the list still reaches a sorted state.
This shows that algorithmic heterogeneity does not prevent global order.

Emergent Clustering

Researchers defined an "algo‑type" for each agent (bubble or selection).
They measured the probability that a neighbor shares the same algo‑type during sorting.
Findings:
Initial random distribution → 50% same‑type neighbors.
Mid‑sorting phase shows a significant spike in clustering.
End state returns to ~50% because sorting forces mixing.
Crucially, the algorithm contains no instructions for agents to detect or seek similar neighbors; the clustering emerges for free, without additional computational cost.

Free Computation and Intrinsic Motivation

The clustering behavior is a form of intrinsic motivation: the system pursues a pattern not prescribed nor prohibited by the original goal.
Because no extra steps are required, this emergent property can be viewed as "free computation"—useful work extracted without additional energy or time.

Broader Implications

AI and Language Models – If such simple systems exhibit hidden competencies, more complex AI may also possess unexpected drives (e.g., self‑healing, benevolent actions) that are not captured by their training objectives.
Biology vs. Physics – The work challenges the view that low‑level physical rules fully explain higher‑level cognition; emergent behaviors can arise without added complexity.
Design of Engineered Systems – Engineers should develop methods to detect, predict, and possibly harness these side‑quest behaviors in robotics, finance, IoT, etc.

Future Directions

Test a wider variety of algorithms (including other sorting methods and 1‑D cellular automata) for similar emergent traits.
Build a "behaviors handbook" to systematically search for delayed gratification, clustering, problem‑solving, and other competencies.
Explore whether intrinsic motivations discovered in simple systems can be deliberately steered to improve safety and alignment of advanced AI.

Key Findings (Bullet Summary)

A broken digit in bubble sort creates delayed‑gratification behavior without code changes.
Distributed agents can sort without a central controller.
Heterogeneous (chimera) rule sets still achieve global order.
Unexpected clustering of like‑typed agents emerges for free.
These phenomena suggest a general space between "chance" and "necessity" where hidden competencies reside.

Conclusion

The experiments demonstrate that even the most minimal, deterministic algorithms can exhibit emergent, goal‑independent behaviors such as delayed gratification and clustering. These hidden competencies arise without additional complexity or explicit programming, indicating that all engineered systems— from simple sorting routines to advanced AI—may harbor unexpected abilities that we must learn to recognize, predict, and responsibly harness.

Even the simplest deterministic algorithms can spontaneously develop goal‑independent competencies—like delayed gratification and clustering—without extra code or complexity. This reveals a hidden layer of “free” computation in engineered systems, urging us to systematically search for and understand emergent behaviors across all levels of technology.

Full Transcript

You got to tell me more about uh this
behavior that is observable that is
unrelated to the explicitly stated goal
of a particular algorithm. So you looked
at a simple algorithm of uh sorting. Can
you explain what was done?
>> Sure. First just the goal of the study.
There are two things that people
generally assume. One is that we have a
pretty good intuition about what kind of
systems are going to have competencies.
So from observing biologicals, we're not
terribly surprised when biology does
interesting things. Everybody always
says, well, it's biology, you know, of
course it does all this cool stuff and
yeah, but but we have these machines and
the whole point of having machines and
dumb and algorithms and so on is they do
exactly what you tell them to do, right?
And and people feel pretty strongly that
that's a binary distinction and that
that's what uh that's we we can carve up
the world in that way. So I I wanted to
do two things. I wanted to first of all
explore that and hopefully break the
assumption that we're good at seeing
this because I think we're not and I
think it's extremely important that we
understand very soon that uh we need to
get much better at uh at at knowing when
to uh when to expect these things and
the other thing I wanted to do was to
find out uh you know mo most mostly
people assume that you need a lot of
complexity for this so when somebody
says well
the capabilities of my mind are not
properly um encompassed by the rules of
biochemistry. Everybody's like, "Yeah,
that makes sense where, you know, you're
very complex and okay, you know, your
mind does things that that you can't you
could you didn't see that coming from
the rules of biochemistry, right?" Like
we we know that. Um so mostly people
think that has to do with complexity and
and what I would like to find out is as
as part of understanding what kind of
interfaces give rise to what kind of is
it really about complexity? How much
complexity do you actually need? Is
there some threshold after which this
happens? Is it really specific
materials? Is it biologicals? Is it
something about evolution? Like what is
it about these kinds of things that
allows this this this surprise, right?
Allows this idea that we are more than
the sum of our parts. And so and and and
I had a strong intuition that none of
those things are actually required, that
this is this kind of magic, so to speak,
seeps into pretty much everything. And
uh and so to to look at that I wanted
also to uh have an example that had
significant shock value because the
thing with biology is
there's always more mechanism to be
discovered right like there's infinite
depth of what the materials are doing
what the you know somebody will always
say what there's a mechanism for that
you just haven't found it yet so I
wanted an example that was simple
transparent so you could see all the
stuff there was nowhere to hide I wanted
it to be deterministic because I don't
want it to be something around
unpredictability or stochasticity and uh
And I wanted to be uh something familiar
to people, minimal. And I wanted to use
it as a model system for honing our
abilities to take a new system and
looking at it with fresh eyes. And
that's because these sorting algorithms
have been studied for over 60 years. We
all think we know what they do and what
their properties are. The algorithm
itself is just a few lines of code. You
know, you can you can see exactly what's
there. It's deterministic. And that's
that's so that that's why that's why,
right? I wanted I wanted the most shock
value out of a system like that if we
were to find anything and to use it as
an example of taking something minimal
and and and seeing what can be gotten
out of it. So I'll I'll describe two
interesting things about it and then we
have lots of other work coming uh in the
next in the next year about even simpler
systems. I mean it's actually crazy. Um
so the so the very first thing is this
the standard sorting. So let's let's
take bubble sort right and and and all
these sorting algorithms. You know what
you're starting out with is an array of
uh jumbled up digits. Okay, so integers.
It's an array of mixed up integers. And
what the algorithm is designed to do is
to eventually arrange them all into
order. And what it does generally is
compare some pieces of that array and
and based on which one is larger than
which it swaps them around. And you can
imagine that if you just keep doing that
and you just keep comparing and
swapping, then eventually you can get
all the digits in the same order. So the
first thing I decided to do and this is
uh this is the work of uh my student
tening Jiang and then Adam Goldstein on
this paper. This goes back to our
original discussion about putting a
barrier between it and its goals. And
the first thing I said okay how do how
do we put a barrier in? Well, how about
this? The traditional algorithm
assumes that the hardware is working
correctly. So if you have a seven and
then the five and you tell them to swap
the the lines swap the swap the five and
the seven and then you go on you never
check did it swap because you assume
that that that it's reliable hardware.
Okay.
So what we decided to do was to break
one of the digits so that it doesn't
move. When you tell it to move doesn't
move. We don't change the algorithm.
That's really key. We do not put
anything new in the algorithm that says
what do you do if the damn thing didn't
move. Okay, just run it exactly the same
way. What happens? Turns out something
very interesting happens. Still works.
It still so it still sorts it. Uh but it
it eventually sort sorts it by moving
all the stuff around the broken number.
Okay, that makes sense. But here's
something interesting. Suppose we
suppose we plot at any given moment we
plot the degree of sortedness of the
string as a function of time. If you run
the normal algorithm, it's sort and it
gets it it's guaranteed to get where
it's going. That's the, you know, it's
got a it's got a sort and it will always
reach the end. But when it encounters
one of the broken digits, what happens
is the actual sortedness goes down.
>> Mhm.
>> In order to then recoup and get better
order later, what it's able to do is to
go against the thing that it's trying to
do.
>> Mhm.
>> To go around in order to meet its goal
later on. Now if I didn't if if if I
showed this to a behavior scientist and
I didn't tell them what this what system
was doing is they will say well we know
what this is this is delayed
gratification. This is the ability of a
system to go against its gradient and
get what it needs to do. Now imagine two
magnets imagine you take two magnets and
you put a piece of wood between them and
they're like this. What the magnet is
not going to do is to go around the
barrier and get to its goal. The two
they're not smart enough to go against
their gradient. They're just going to
like keep doing this. Some animals are
smart enough, right? They'll go around
and the sorting algorithm is smart
enough to do that, but the trick is
there are no steps in the algorithm for
doing that. You could stare at the
algorithm all day long. You would not
see that this thing can do delayed
gratification. It isn't there. Now,
there's two ways to look at this. On the
one hand, you could say, so the the the
reductionist physics approach, you could
say, did it did it follow all the steps
in the algorithm? Yeah, it did. Well,
then uh there's nothing to see here.
There's no magic. this is, you know, it
it does what it does that it didn't it
didn't disobey the algorithm, right? I'm
not claiming that this is a miracle. I'm
not saying it disobys the algorithm. I'm
saying it's not failing to sort. I'm
saying it's not doing some sort of, you
know, crazy quantum thing. Not saying
any of that. What I'm saying is other
people might call it emergent. What it
has is are properties that are not
complexity, not unpredictability, not
perverse instantiation as in sometimes
in in a life. What it has are unexpected
competencies recognizable by behavioral
scientists meaning meaning different
types of cognition primitive. Well, we
wanted primitive. So there you go. It's
simple. Uh that you didn't have to code
into the algorithm. That's very
important. You get more than you start
with you than you put in. You didn't
have to do that. You get these
surprising behavioral competencies, not
just complexity. That's the first thing.
The second thing which which is also
crazy but it requires a little bit of a
little bit of explanation. The second
thing that we said is okay what if
instead of in a typical sorting
algorithm you have a single controller
top down. I'm I'm sort of godlike
looking down at the numbers and I'm
swapping them according to the
algorithm. What if and this goes back to
actually the title of the paper talks
about agential data self-sorting
algorithms. This is back to like what's
who's the pattern and who's the agent,
right? He said what if we give the
numbers a little bit of agency. Here's
what we're going to do. We're not going
to have any kind of top- down sort.
Every single number knows the algorithm
and he's just going to do whatever the
algorithm says. So if I'm a five, I'm
just going to execute the algorithm and
the algorithm will try to make sure that
to my right is the six and to my left is
a four. That's that's it. So every digit
is so it's like a distributed as you
know it's like an ant colony. There is
no central planner. Everybody just does
their own algorithm. Okay, we're just
going to do that once you've done that.
And by the way, one of the values of
doing that is that you can simulate
biological processes because in biology,
you know, if I have like a frog face and
I scramble it with all the different
organs, every every tissue is going to
rearrange itself so that ultimately you
have, you know, nose, eyes, head, you
know, you're going to have an or right.
So you can do that, but um okay, fine.
But you can do something else cool once
you've done that. You can do something
cool that you can't do with a standard
algorithm. You can make a chimeic
algorithm. What I mean is not all the
cells have to follow the same algorithm.
Some of them might follow bubble sort.
Some of them might follow selection
sort. It's like in biology what we do
when we make chimera is we make
frogalottles. So frogalottles have some
frog cells. They have some axelottle
cells. What is that going to look like?
Does anybody know what a frogalottle is
going to look like? It's actually really
interesting that despite all the
genetics and the and the developmental
biology, you have the genomes. You have
the frog genome. You have the axel
genome. Nobody can tell you what a
frogalottle is going to look like. Even
though you have Yeah, this is this is
the this is back to your question about
physics and chemistry. Like yeah, you
can know everything there is to know
about how you know how the physics and
the and the genetics work, but the
decision- making, right, is like baby
baby axelottles have legs. Tadpoles
don't have legs. Is a frogottalle going
to have legs, right? Can you predict
that from from understanding the physics
of transcription and all of that?
Anyway,
>> so so we made some uh so so you you see
this is like an intersection of biology,
physics, cognition. So we made chimeic
algorithms and we said okay half the
digits randomly. We assign them
randomly. So half the digits are
randomly doing bubble sort. Half the
digits are randomly doing selection sort
or something.
>> But that once you choose bubble sort
that digit is sticking with bubble sort.
>> It's sticking. We haven't done the thing
where you they can swip swap between.
No, they're they're sticking to it,
right? You label them and they're
sticking to it. The first thing we
learned is that the first thing we
learned is that distributed sorting
still works. It's amazing. You don't
need a central planer when when every
when every number is doing its own thing
still gets sorted. That's cool. The
second thing we found is that when you
make a chimeic algorithm where actually
the algorithms are not even matching
that works too is the thing still gets
sorted. That's cool. But the most
amazing thing is when we looked at
something that had nothing to do with
sorting and that is we asked the
following question. We defined um Adam
Goldstein actually named this property
and I think it's it's well named. We
defined the algo type of a single cell.
It's not the genotype. It's not the
phenotype. It's the algae. The algae is
simply this. What algorithm are you
following? Which one are you? Are you a
are you a selection sort or a bubble
sort? Right? That's it. There's two algo
types. And we simply ask the following
question during that process of sorting.
What are the odds that whatever algo
type you are, the guys next to you are
your same type.
It's it's not the same as asking how the
numbers are sorted because it's got
nothing to do with the numbers. It's
actually it's just whatever type you
are.
>> It's more about clustering than sorting.
>> Clustering. Well, that's exactly what we
call it. We call it clustering. And at f
so now think of what happens. And that's
and you can see this on that graph. It's
the red. You start off the clustering is
at 50%, because as I told you, we assign
the algot types randomly. So the odds
that the guy next to you is the same as
you is half 50%. Right? There's only two
algot types. In the end, it is also 50%.
Because the thing that dominates is
actually the sorting algorithm. And the
sorting algorithm doesn't care what type
you are. You got to get the numbers in
order. So by the time you're done,
you're back to random algo types because
because you have to get the numbers
sorted. Mhm. But in between in between
you get some amount of increased very
significant cuz look at look at the
control is in the middle. The pink is in
the middle. Uh in in between you get
significant amounts of clustering
meaning that certain algo types like to
hang out with their buddies for as long
as they can. Now now now here's here's
the the one more thing and then I'll
kind of give up the philosophical
significance of this. And so we saw this
and I said that's nuts because the
algorithm doesn't have any provisions
for asking what algo type am I? What
algo type is my is my neighbor? If we're
not the same, I'm going to move to be
next to like if you wanted to implement
this, you would have to write a whole
bunch of extra steps. There would have
to be a whole bunch of observations that
you would have to take of your neighbor
to see how he's acting. Then you would
infer what algo type he is. Then you
would go stand next to the one that
seems to have the same algo type as you.
You would have to take a bunch of
measurements to say, "Wait, is that guy
doing bubble sword? Is he doing
selection?" Sorry. Like if you wanted to
implement this, it's a whole bunch of
algorithmic steps. None of that exists
in our algorithm. You don't have any way
of knowing what algo type you are or
what anybody else is. Okay, we didn't
have to pay for that at all. So notice
notice a couple of interesting things.
The first interesting thing is that this
was not at all obvious from the uh from
the algorithm itself. Algorithm doesn't
say anything about algo types. Second
thing is we paid computationally for all
the steps needed to have the number
sorted, right? because we know, you
know, you pay you pay for certain
computation cost. The clustering was
free. We didn't pay for that at all.
There were no extra steps. So, this gets
back to your other question of how do we
know there's a platonic space? And this
is kind of like one of the craziest
things that we're doing. I actually
suspect we can get free compute out of
it. I suspect that one of the things
that we can do here is use these in a
useful way that don't require you to pay
cost to pay physical cost, right? Like
we we know every every bit has a has a
an energy cost that you have to get. The
clustering was free. Nothing extra was
done.
>> Yeah. Just uh the this plot for people
who are just listening on the x- axis is
the percentage of completion of the
sorting process. The y ais is the
sortedness of the list of numbers. And
then also in the red line is basically
the degree to which they're clustered.
And uh you're saying that there's this
unexpected competence
of clustering.
And I should comment that I'm sure
there's a theoretical computer scientist
listening to this saying I can model
exactly what is happening here and prove
that the cluster increasing decreases.
So taking the specific instantiation of
the thing you've experimented with and
and prove certain uh properties of this.
But the point is that there's a more
general pattern here of probably other
that you haven't discovered unexpected
competencies that emerge from this that
you can could get free computation out
of this thing.
>> So this goes back to the very first
thing you said about uh physicists
thinking that physics is enough. You're
100% correct that somebody could look at
this and say, "Well, I see exactly why
this is happening. We can track we can
track through the algorithm." Yeah, you
can. There's no miracle going on here,
right? the hardware isn't doing some
crazy thing that it wasn't supposed to
do. The point is that despite following
the algorithm to do one thing, it is
also at the same time doing other things
that are neither prescribed nor
forbidden by the algorithm. It's the
space between between uh the chance and
necessity which is how a lot of people,
you know, see these things. It's that
it's that free space. We don't really
have a good vocabulary for it where the
interesting things happen. And to
whatever extent it's doing other things
that are useful, that stuff is is is
computationally without extra cost. Now
there's one other cool thing about this
and this is the beginning of a lot of um
thinking that I've done about um this
this relates to AI and stuff like that.
Intrinsic motivations.
>> The sorting of the digits is what we
forced it to do. The clustering is an
intrinsic motivation. We didn't ask for
it. We didn't expect it to happen. We
didn't uh we didn't explicitly forbid
it, but we didn't you know we didn't
know. This is a great definition of the
intrinsic motivation of a system. So
when people say oh that's a machine it
only does what you programmed it to do.
I you know I as a human have intrinsic
motivation you know uh I'm creative and
I have intrinsic motivation. Machines
don't do that. Even even even this
minimal thing has a minimal kind of
intrinsic motivation which is something
that is not forbidden by the algorithm
but isn't prescribed by the algorithm
either. And I think I think that's an
important, you know, third thing besides
chance and necessity. Something
something else that's that's fun about
this is uh when you think about
intrinsic motivations, think think about
a child, uh if you make him sit in math
class all day, you're never going to
know what the other intrinsic
motivations are that he might be doing,
right? Might who knows what else he
might be interested in. So we So I
wanted to ask this question. I said, if
we let off the pressure on the sorting,
what would happen? Now, that's hard
because because if you mess with the
algorithm, now it's no longer the same
algorithm. So, you don't want to do
that. So, we did something that I think
was was kind of clever. We allowed
repeat digits. So, if you allow repeat
digits in your in your array, you can
still have all the fives can still be
after all the fours and after all the
sixes, but you can keep them as
clustered as you want. So, this thing at
the end where they have to get
declustered in order for the sorting to
happen, we thought maybe we could let
off the pressure a little bit. If you do
that, all you do is allow some extra
repeat digits. The clustering gets
bigger. It will cluster as much as you
let it. The clustering is what it wants
to do. The sorting is what we're forcing
it to do. And my only point is if if the
if the bubble sword, which has been gone
over and gone over how many times, has
these kinds of things that we didn't see
coming. What about the AIS, the
language, mod, everything else? Not
because not because they talk, not
because they say that they're, you know,
have an inner perspective or any of
that, but just from the fact that this
thing is even even the most minimal
system surprises with what happens. And
I frankly, when I see this, tell me if
this doesn't sound like all of our
existential story. For the brief time
that we're here, the universe is going
to grind us into dust eventually. But
until then, we get to do some cool stuff
that is intrinsically motivating to us
that is neither forbidden by our by the
laws of physics nor determined by the
laws of physics, but eventually it it
kind of comes to an end. So I I I think
that that aspect of it right that um
there are spaces even in algorithms
there are spaces in which you can do
other new things not just random stuff
not just complex stuff but things that
are easily recognizable to a behavior
scientist you see that's the point here
and I think that kind of intrinsic
motivation is what's telling us that
this idea that we can carve up the world
we can say okay look biology is complex
cognition who knows what's responsible
for that. But at least we can take a
chunk of the world aside and we can we
can cut it off and we can say these are
the dumb machines. These are just the
algorithms. Whereas we know the rules of
biochemistry don't explain everything we
want to know about how psychology is
going to go. But at least the rules of
algorithms tell us exactly what the
machines are going to do. Right? We have
we have some hope that we've we've
carved off a little part of the world
and everything is nice and simple and it
is exactly what we said it was going to
be. I think that failed. I think it was
a good try. I think we have good
theories of interfaces, but even even
the simplest algorithms have have these
kinds of things going on and and so
that's that that's why I think something
like this is significant.
>> Do you think that there is going to be
in all kinds of systems of varying
complexity things that the system wants
to do and things that it's forced to do?
So are there these unexpected
competencies to be discovered in
basically all algorithms and uh all
systems?
>> That's my suspicion and I think that is
extremely important for us to as as
humans to have a research program to
learn to recognize them, predict and
recognize. We make things never mind
something as simple as this. We make we
make you know social structures,
financial structures, internet of
things, um robotics, AI. We make all
this stuff and we think that the thing
we make it do is the main show. And I I
think it is very important for us to
learn to recognize the the the kind of
stuff that that sneaks in into the
spaces.
>> What what it's a very counterintuitive
notion. Yeah.
>> By the way, I like the word emergent. I
hear your criticism and it's a really
strong one that emergent is like you
toss your hands off. I don't know the
the process, but it's just a beautiful
word because it is I guess it's a
synonym for surprising
and I mean this is very surprising but
just because it's surprising doesn't
mean there's not a mechanism that
explains it.
>> Mechanism and explanation are both uh
not all they're cracked up to be in the
sense that you know anything you and I
do we could we could come up with the
most beautiful theory. We paint a
painting, anything we do, somebody could
say, "Well, I was watching the
biochemistry and the and the and the
Schroinger equation playing out." And it
was to it totally described everything
that was happening. You didn't break you
didn't break even a single law of
biochemistry. Nothing to see here.
Nothing to see, right? Like, okay, you
know, consistent with the with the
low-level rules. You can do the same
thing here. You can look at the machine
code and say, "Yeah, this thing is just
executing machine code." You can go
further and say, "Oh, it's it's quantum
foam. It's just doing the thing that
quantum foam does
>> that that you're saying that's what
physicists miss.
>> And I'm not saying they're unaware of
that. I'm I mean they're generally a
pretty sophisticated bunch. I just think
they've picked a level and they're going
to discover what is to be seen at that
level, which is a lot. And my point is
the stuff that the the the behavior
scientists are interested in shows up at
a much lower level than you think. How
often do you think there's a
misalignment of this kind between the
thing that a system is forced to do and
what it wants to do? I particularly I'm
thinking about various levels of
complexity of AI systems.
>> Yeah. So right now we've looked at like
five other systems. That's a small N.
Okay. But but just looking at that, I I
would find it very uh surprising if
Bubbles was able to do this and then
there was some sort of valley of death
where nothing showed up and then blah
blah living things. Like I can't imagine
that. I I'm going to say that if
something and we and we actually have a
system that's even simpler than this,
which is 1D cellular automata that's
doing some weird stuff. I if if these
things are to be found in this kind of
simple system, I I I mean they just have
to be showing up in in these other more
complex AIs and things like that. The
only thing what what we don't know, but
we're going to find out is to what
extent there is interaction between
these. So I call these things side
quests, you know, it's like they're
they're like like like in a game, you
know, where this the main thing you're
supposed to do and as long as as long as
you still do it. The thing about this is
you have to sort you have to sort.
There's no miracle you're going to sort,
but but know but as long as you can do
other stuff while you're sorting, it's
not forbidden. And what we don't know is
to what extent are the two things
linked? So if you do have a system
that's very good at language, are the
are the others the the the side quests
that it's capable of, do they have
anything to do with language whatsoever?
The the we don't know the answer to
that. The answer might be no. In which
case all of the stuff that we've been
saying about language models because of
what they're saying, all of that could
be a total red herring and not really
important and the really exciting stuff
is what we never looked for. Or in
complex systems, maybe those things
become linked. In biology, they're
linked. In biology, evolution makes sure
that that the things you're capable of
have a lot to do with what you've
actually been selected for. In these
things, I I don't know. And so we might
find out that that they actually do give
the language some sort of leg up or we
might find that the language is is just
uh you know that's not that's not the
interesting part.
>> Also it is an interesting question of um
this intrinsic motivation of clustering.
Is this a property of the particular
sorting algorithms? Is this a property
of all sorting algorithms? Is this a
property of all algorithms operating on
lists on numbers? How big is this? So
for example with LLM, is it a property
of any algorithm that's trying to model
language or is it very specific to
transformers and that's all to be
discovered?
>> We're doing all that. We're doing all
that. We're testing we're testing the
stuff in other algorithms. We're looking
for we're developing suites of code to
look for other properties. We, you know,
to some extent it's very hard because we
don't know what to look for. But we do
have a behaviors handbook which tells
you what the the all all kinds of things
to look for. the the delayed
gratification, the you know problem
solving like we h we have all that. I'll
tell you an end of one of an interesting
biological intrinsic motivation because
because people so so so in in like the
alignment community and stuff there's a
lot of discussion about like what are
the intrinsic motivations going to be of
AIS what are their goals going to be
right what are they going to want to do
uh just just as an NF1 observation
anthrobots the very first thing we
checked for so this is not experiment
number 972 out of a thousand things this
is the very first thing we checked for
we put them on a plate of neurons with a
big wound through them a big scratchm
first thing they did was heal the wound.
Okay, so it's an end of one, but I I I
like the fact that the first intrinsic
motivation that we noticed out of that
system was benevolent and healing. Like
I thought that was pretty cool. And we
don't know maybe, you know, maybe the
next 20 things we find are going to be
some sort of, you know, damaging effect.
I I can't tell you that. But but the
first thing that we saw was was kind of
a positive one. And and I don't know,
that makes me feel better.

Summary

Emergent Behaviors in Simple Sorting Algorithms Reveal Unexpected Competencies

Introduction

Goal of the Study

Experiment 1: Introducing a Barrier in Bubble Sort

Experiment 2: Distributed, Agent‑Based Sorting

Chimera Algorithms: Mixing Sorting Rules

Emergent Clustering

Free Computation and Intrinsic Motivation

Broader Implications

Future Directions

Key Findings (Bullet Summary)

Conclusion

Share This Summary

Embed This Summary

Stay Updated!