Joseph Robert Turcotte PRO

I love tiny models.

AI wasn't helped in making hands with a particular number of fingers without a coherent way of achieving that from the start and a bunch of rather unfiltered data. It's getting better in many regards. For example, AI should have a rather good understanding of anatomy, and should probably start with bones of various animals, like anatomy courses for human students. Then, you could tell it the numbers of fingers, or to make a wing of a particular type. I'm sure lots of people have mentioned this before, but it's a good example.

That's an interesting way of accomplishing it. I would assume enough filtering would accomplish much the same with huge datasets for multi-modal AI, but focusing primarily on improving and even simplify existing scenarios with AI should be largely the goal of smol models and similar schemes to reduce the size and complexity of interactions while preserving precision and value. The focus could and should help in particular scenarios, but I admittedly don't know the particulars well enough, so I'd be largely speculating on what others may or may not be able to accomplish with current limitations.

I hope we can learn to agree in the future that anything with a degree of confidence is trying at an art rather than pure science, but that it will highlight how much of our work was art anyway.

It shouldn't have to explain to you why it's better. Better should be a rather floaty standard, as in Stable Diffusion. In general, better means better in all metrics, but if you need specific metrics, it should know to keep all betterment that fits rather general standards better. I don't think it does. If it can know it's better, it shouldn't have to explain that it's done that, no matter if you prefer black boxes or not. And this is how it goes: Electronics engineers feed models their goals, and they produce circuits in shapes they can't explain, but that work. This is happening now. Most of our science is of this type, where we simply accept that things fit without knowing fully why.

What happens when you tell something like Stable Diffusion to build a human hand without any advice on what actually composes a human hand and a bunch of biomimetic parts with materials science to choose from? When does it invent a better hand accidentally?

If I spend enough time, I should be able make a bot from scratch that's 20 times smaller, using the same code and structure. Why am I saying this? Am I going to do it? No, but everyone should know that most of what people do is throwing a ton of information at bots to work out for themselves algorithmically. We need more experimental bots, particularly to skip a few steps toward getting the same answer. So, I'm always glad to see work of this sort, whether it's trying different datasets with different LLMs, or whatever.

When does the planned context become the signifier of that context in the code itself? When something is stable in code. Even having to recover, or being able to, means it's storing far too much about context without getting to the context itself. All language needs the same simplification. Or maybe I just don't see reflexivity in AI yet. Maybe I don't see it building itself with awareness of what it is to others, unlike NASNet.