I stumbled upon the problem of fail2ban not banning after I had moved my ssh server to non standard port (let’s say 22022).

You already know that it is not so smart to leave SSH running on your servers on default port and accessible from every internet address (ie. no firewall restrictions, no host allow/deny).. but in real world it happens to do so since, let’s say, you have no static IP, you have no access to firewall rules and so on.