The reason games sometimes perform lower with CPUs with more cores (6 vs 4) is mostly due to the optimizations software developers compile games for. These optimizations have to assume a certain amount of L2/L3 caches. So when you throw the game over bigger CPUs (with more cores) that game has to share the same caches, therefore the level of efficiency reduces because more cores are reading/writing from/to the same cache space.

In principle, if the game developers were nice enough to us, we could have at least N versions of the game, depending on the PC we use.

Some games are not affected by these problems and actually scale very well with core count. These usually means they are either bound to some other resource (GPU for example) or because the amount of aggressiveness to CPU caches is not enough to slow down core pipeline execution.

Goot post by the way. Going to upvote via @minnowsupport!