This is not something that is supported by cloudformation template out of the box. You can technically do this if you remove Cognito bits and you probably will have to replace ALB with something else as well, and launch another ec2 instance to work as a ssh tunnel bastion host in the same VPC.
But honestly its almost certainly not worth your time given that the cheapest domain in Route53 can be registered for ~$5 and SSL cert is free, unless you want to do it for fun :)