I think you'll be better off bumping up the resources per task a bit more before worrying about scaling the number of tasks
In the
cloudformation example stack they use 4 CPU / 16GB RAM, and in the
terraform example stack they use 2 CPU / 16GB RAM.
Anecdotally, I've been using 4 CPU / 16GB RAM, autoscaling the number of tasks between 1-10, with the policy metric targeting 40% average memory utilization – this has worked well for a decently sized organization in my experience