Controller not being detected

What do you mean by "just thinks"? You have to select the DS4 Input Handler, it won't magically be select if you just connect the controller. If you select it, then your Keyboard won't work at all on the game, even if DS4 also doesn't.

Make sure you don't have per game configuration overriding your main configuration, if you do, then configure your input on that per game configuration option (right click on game and clicking on configuration is per game configuration).