Puzzles: sometimes (rarelyBTW) the line in reply to solution's first move isn't the most challenging

@StrongPaleAle said in #9:
> Ok, "a loss is a loss" ...but why did the engine choose the fifth best line as the solution to deliver on the puzzle?

When several moves are close, what is seen as the best line will depend on different things such as exact engine version and depth it searches at. If using multithreading, it can give different results even at fixed depth with the same version.

The only reasonable way to properly fix these edge cases would be to have an engine predicting potential human moves (something along the lines of maia), then combining it with Stockfish using some kind of averaging search to maximize the odds of human failure until the end of a forced sequence. But that's a lot of work (and complication of actually running puzzle generation) for the benefit.

Don't get me wrong, I would like to see it done. But it's very understandable that it's far from a priority.

I'd argue there would be less work and more benefit looking at introducing other types of puzzles, such as expanding the "save equality puzzles" they exist but are rather hidden and limited, or introducing positions with multiple solutions (each of which is rated as a success) , as while that limits strict categorization of what a puzzle is about, it does open more type of tricky positions ; or even simply have completely normal positions (quiet and tactical), and just be rated on how close the picked move is in acpl to the engine-determined best move.

Or introducing variant puzzles (especially 960 puzzles).

This is veery good, thank
you all for the feedback !!
SPA

Another way to deal with this (and are we sure this isn't already being done?) is to allow multiple puzzles to be generated from the same starting position. If different continuations by the opposing side make for puzzles that meet the criteria then make a puzzle for each of them.

@kingdave2 said in #13:
> Another way to deal with this (and are we sure this isn't already being done?) is to allow multiple puzzles to be generated from the same starting position. If different continuations by the opposing side make for puzzles that meet the criteria then make a puzzle for each of them.

Not sure I understand what you mean by "starting position". The position for the human to work on, is usually itself selected from games at the position successor to the move blunder, that has a very peaked solution for its first solution move, if I got that right reading posts here and doing puzzles on lichess for the last 5 years (soon).

So, it might not be about that starting position? Or are you wanting less peaked first move solution profiles to be picked from games,. Or maybe not only consider the game blunder, but other non best moves (assuming we trust engine to tell that).

I often find that things are clear for the first solution more to find, because that is rigged that way. I think within further depth of the solution (this might not be about the op specific non testing SF behavior problem), there might be more room for such idea about allowing more than one path (or position ... along the way).. But that would not fit well, your proposition...

Oh, I see, consider the same starting position, but not the same puzzle id, depending on the reply nature. which could have different difficulty levels. sorry. I often have to retrace the reasoning behind concise posts.... a way to compensate for my ignorance, reasoining to the rescue.. i hope I got it.

@dboing Well, that was a good read! I think you got what I was saying in the end. Reading your in depth analysis though made me think that my idea might not be straightforwardly implemented. The phenomenon that it would be nice to address is when the puzzle response is the best per the engine, but not the most challenging for people. Maybe there is a difficult to find mate in 5, but instead it plays into an easy to find mate in 6. The problem is there probably isn't a simple formula that captures the idea of "more challenging." Also, any alternative version of the puzzle would still have to have the characteristic that at each turn for the solver there is one superior move.

Yes, tactics puzzles are turn by turn: success or failure things to figure out, well for a short puzzle session, and not a graduation of the possible moves alternatives, which would need a lot of trust on engine being able to not just find best move, but have even more resolution than that on flatter profiles.

And yes this thread was about challenging aspect being sustained. or engine replies being testing all along the solution definition that calls a success or failure (if keeping that part of the system).

There is another problem, I value the subjective themes in the puzzle system more than how hard a puzzle is. I am curious about the tactical theory, and found that I could learn a lot there.

Long hard puzzles throughout, make for poor separation of themes, as it is enough that one turn be failed for the whole puzzle to be failed, and all its themes, whether used appropriately or not, count as failure for each.

So it might depend the purpose of the puzzle activity. Hard for hard, or radar plot hard, with significance of the performance rating per names themes (which, I should say, have pretty good definitions upon hover as a set of themes, or getting there).

Yet, all sort of constraints around chess mobility rules make for fine sessions of head banging against some mind's eye wall...:)

@Akbar2thegreat said in #5:
> What we may think of as challenging may not be the case with engine.

No, it's because it has no concept at all of what is "more challenging," since it's all the same to it.