You would have to set up two separate pools of php "workers" in your php-fpm.conf, each using a different php_defines as well as a different port. One would have APC enabled, one would not. How to enable or disable APC would depend on if you built it into the PHP executable or not. If it is built in, you'd add apc.enabled=0 in one php_defines to disable it in that worker pool. Otherwise, you'd remove the extension=apc.so from your default .ini file and move it into one of the php_defines to enable it in that worker pool.
In nginix, you should be able to set which worker pool gets used on a per-location basis by putting the correct port in fastcgi_pass. You could test if its working or not by creating a small script calling phpinfo() in each location and checking the output for APC.