> don’t need a universally justified definition, I’m just looking for an objective, scientific one. A definition that would help us say for sure that a particular cognition is or isn’t a product of reason.
Unfortunately, you won't get one. We simply don't know enough about cognition to create rigourous definitions of the type you are looking for.
Instead, this paper, and the community in general are trying to perform practical capability assessments. The claim that the GSM8k measures "mathematical reasoning" or "logical reasoning" didn't come from the skeptics.
Alan Turring didn't try to define intelligence, he created a practical test that he thought would be a good benchmark. These days we believe we have better ones.
> I just don’t find this kind of critique specific enough to have any sort of scientific validity. IMHO
"Good cognition" seems like dismisal of a definition, but this is exactly the definition that the people working on this care about. They are not philosphers, they are engineers who are trying to make a system "better" so "good cognition" is exactly what they want.
The paper digs into finding out more about what types of changes impacts peformance on established metrics. The "noop" result is pretty interesting since "relevancy detection" isn't something we commonly think of as key to "good cognition", but a consequence of it.
Unfortunately, you won't get one. We simply don't know enough about cognition to create rigourous definitions of the type you are looking for.
Instead, this paper, and the community in general are trying to perform practical capability assessments. The claim that the GSM8k measures "mathematical reasoning" or "logical reasoning" didn't come from the skeptics.
Alan Turring didn't try to define intelligence, he created a practical test that he thought would be a good benchmark. These days we believe we have better ones.
> I just don’t find this kind of critique specific enough to have any sort of scientific validity. IMHO
"Good cognition" seems like dismisal of a definition, but this is exactly the definition that the people working on this care about. They are not philosphers, they are engineers who are trying to make a system "better" so "good cognition" is exactly what they want.
The paper digs into finding out more about what types of changes impacts peformance on established metrics. The "noop" result is pretty interesting since "relevancy detection" isn't something we commonly think of as key to "good cognition", but a consequence of it.