You're seriously insisting that the definition of novel in science only includes things that thousands of people have worked on for decades and haven't solved?
An example problem includes the "Erdos set" problems (see problem 124).
But also, LLMs have solved Olympia problems, see the results of IMO 2025. You can say that these are not interesting or challenging problems, but in the context of the original discussion, I don't think you can discount them as "novel". This is what the original comment said:
> Not so much amazing as bewildering that certain results are possible in spite of a lack of thinking etc. I find it highly counterintuitive that simply referencing established knowledge would ever get the correct answer to novel problems, absent any understanding of that knowledge.
I think in this context, it's clear that IMO problems are "novel" - they are applying knowledge in some way to solve something that isn't in-distribution. It is surprising that this is possible without "true understanding"; or, alternatively, LLMs do have understanding, whatever that means, which is also surprising.
An example problem includes the "Erdos set" problems (see problem 124).
But also, LLMs have solved Olympia problems, see the results of IMO 2025. You can say that these are not interesting or challenging problems, but in the context of the original discussion, I don't think you can discount them as "novel". This is what the original comment said:
> Not so much amazing as bewildering that certain results are possible in spite of a lack of thinking etc. I find it highly counterintuitive that simply referencing established knowledge would ever get the correct answer to novel problems, absent any understanding of that knowledge.
I think in this context, it's clear that IMO problems are "novel" - they are applying knowledge in some way to solve something that isn't in-distribution. It is surprising that this is possible without "true understanding"; or, alternatively, LLMs do have understanding, whatever that means, which is also surprising.