• 2 Posts
  • 1.17K Comments
Joined 2 years ago
cake
Cake day: June 18th, 2023

help-circle

  • with an assumed entropy of about 5 bits per English word. A 120 wpm typing speed therefore translates to 600 bits per minute, or 10 bits per second. A 160 wpm speaking speed translates to 13 bits/s.

    The problem here is that the bits of information needs to be clearly defined, otherwise we are not talking about actually quantifiable information. Normally a bit can only have 2 values, here they are talking about very different types of bits, which AFAIK is not a specific quantity.

    the human brain tends to trick itself into perceiving a much higher complexity that it is actually processing

    This is of course a thing.



  • OK I don’t think information theory is actually needed for that. Just a bit of above average intelligence apparently.
    Yes it’s true some use the term entropy, instead of just the statistical amount of combinations, and obviously forcing a special character instead of just having it as an option, makes the number of possibilities lower, decreasing the uncertainty, which they then choose to call entropy. Which counter intuitively IMO is called increased entropy.





  • I’m not sure if you are making a joke, or also making a point. But boy that XKCD is spot on. 😋 👍
    I think within it’s field thermodynamics works, but it’s so widely abused outside the field I’ve become sick of hearing about it from people who just parrot it.
    I have not seen anything useful from information theory, mostly just nonsense about information not being able to get lost in black holes. And exaggerated interpretations about entropy.
    So my interest in information theory is near zero, because I have discarded it as rubbish already decades ago.



  • I get your argument now.

    Thanks. ;)

    Do note that this entropy is about information theory and not thermodynamics

    https://en.wikipedia.org/wiki/Information_theory

    A key measure in information theory is entropy.

    Meaning it’s based on thermo dynamics.

    And incidentally I disagree with both. Information theory assumes the universe is a closed system, which is a requirement for thermodynamics to work. which AFAIK is not a proven fact regarding the universe and unlikely IMO.

    2nd law of thermodynamics (entropy) is not a law but a statistical likelihood, and the early universe does not comply, and the existence of life is also a contradiction to the 2nd law of thermodynamics.

    I have no idea how these ideas are so popular outside their scope?


  • I suppose it can, but just calling it bits is extremely misleading. It’s like saying something takes 10 seconds, but only if you are traveling 90% at the speed of light.
    It such extremely poor terminology, and maybe the article is at fault and not the study, but it is presented in a way that is moronic.

    Using this thermodynamics definition is not generally relevant to how thought processes work.
    And using a word to mean something different than it usually does BEFORE pointing it out is very poor terminology.
    And in this case made them look like idiots.

    It’s really too bad, because if they had simply stated we can only handle about 10 concepts per second, that would have been an entirely different matter, I actually agree is probably right. But that’s not bad IMO, that’s actually quite impressive! The exact contrary of what the headline indicates.


  • Oh no this is not annoying, this is a very interesting question.
    I suppose with the crow, it doesn’t need to understand volume of water and rocks displacing it, but merely has a more basic understanding that adding rocks raise the water, or maybe even just makes the food easier to get at.
    So I suppose we can agree that there are multiple levels of understanding.
    But still the crow must have observed this, unless it actually figured it out? And some thought process must have led it to believe that dropping stones in the water might have the desired effect.
    Now even if the crow had observed another crow doing this, and seen this demonstrated. Ir mist have had a thought process concluding that it could try this too, and perhaps it would work.

    But there are other situations that are more challenging IMO, and that’s with LLM, how do we decide thought and understand with those.
    LLM is extremely stupid and clever at the same time. With loads of examples of them not understanding the simplest things, like how meany R’s are in Strawberry, and the AI answering stubbornly that there are only 2! But on the other hand being able to spell it out and count them, then being able to realize that there are indeed 3, which it previously denied.

    IMO animal studies are crucial to understand our own intelligence, because the principle is the same, but animals are a simpler “model” of it so to speak.
    It seems to me that thought is a requirement to understanding. You think about something before you understand it.
    Without the thought process it would have to be instinctive. But I don’t think it can be argued that crows dropping rocks in water is instinctive.
    But even instinctive understanding is a kind of understanding, it’s just not by our consciousness, but by certain behavior traits having an evolutionary advantage, causing that behavior to become more common.

    So you are absolutely right that thought is not always required for some sort of “understanding”. which is a good point.
    But what I meant was conscious understanding as in really understanding a concept and for humans understanding abstract terms, and for that type of understanding thought is definitely a requirement.





  • Mostly philosophical, but since I’m also a programmer, I’ve always had the quantized elements in mind too.

    In the year 2000 I estimated human level or general/strong AI by about 2035. I remember because it was during a very interesting philosophy debate at Copenhagen University. Where to my surprise there also were a number of physics majors.
    That’s supposed to be an actually conscious AI. I suppose the chances of being correct were slim at the time, but now it does seem to be more likely than ever.