Assuming by "access point" you mean wireless access points, then in answer to your question, yes this would work and no there's no need to have a wireless controller, however in my experience it makes your life a lot easier. If you need to change the SSID, for instance, you will have to do it 20 times. Also, depending on the authentication method, you will have to manage them individually at each change.
If you have many wireless access points in your area you may also find it hard to manage the channel distribution due to the way most access points will try to autoconfigure themselves, and in busy areas manual channel configuration can result in other access points automatically reconfiguring themselves to the same channel and causing you problems.
All of this will culminate in a management headache for you in the long run, especially if it will be a heavily used wireless network
With a wireless controller you'd be able to manage all this centrally, so changing the SSID or authentication details will be as simple as logging in to the controller and changing it once. As all the access points are managed centrally it will also centrally manage the channels assigned to each point so that they avoid each other as best they can, if one access point starts conflicting with an external access point the controller will manage this for you automatically too.