My public IP address changes every 24 hours which is normal, but my question is: how can I make it static or what setting do I need to change so I don't have to re-allow and recreate a new security group for my new public IP address in AWS EC2?
A dynamic IP address is fairly common in many countries. So is carrier grade NAT, which means many people share the same IP address.
You don't need to create a new security group; you just need to add your new IP address to the existing security group and ideally need to remove all others from the security group. There are scripts you can find online that automate this for you, such as this one. You could extend it to remove old IP addresses using the AWS CLI / SDK. An EC2 instance can have five security groups, I tend to have one dedicated to home IP on various ports, one for my CDN's IPs, and then a "misc" SG for other things. This is useful as SGs have a limit of around 50 rules per group, and keeping things together makes them easier to organise.
Alternately, if your IP addresses are all in a given CIDR range or set of ranges, you could add those to your security group.
Another option is to remove the IP address restriction, which decreases security somewhat, but if you keep your private key private maybe that's good enough. That would open you up to brute force login attempts, and if any vulnerabilities are found in SSH or SSH is poorly configured that could let people into your server, so I wouldn't recommend it but it is an option.
You can sidestep the open IP address issue by using AWS Systems Manager Session Manager.
It logs you in to your EC2 instances through the AWS CLI. How AWS routes your connection internally is a bit of a black box, but it does permit you to keep those ports closed to the public Internet.
To use things like scp or rsync, you can enable ssh connections through the AWS CLI following this documentation.
I'd avoid using another commercial service for a VPN, and instead either
Run a VPN server like OpenVPN on your own EC2 instance in your VPC, or
Since you're already paying for AWS services, use the AWS Client VPN https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/what-is.html
Both will let you connect to your remote host securely, with certificates rather than passwords, and you can leave your security group allowing unrestricted access in on port 1194/udp.
Optionally you could lookup all the IPs owned by your ISP and add the entire blocks to your inbound security group.
Another option is to upgrade your residential grade ISP connection to one with static IPv4 addressing, at some cost, if they offer it, or change to an ISP that does.
Finally, consider IPv6 connectivity. This is relatively simple at the AWS side, but again depends on your ISP supporting it.... not all do.
Some commercial VPNs will offer a solution that gives your traffic a consistent network egress. I've used that solution commercially, especially as many workers previously connecting through the corporate network infrastructure transitioned to work from home arrangements but still needed access to the same cloud resources.
This is likely overkill for anything on the scale of a personal project.